Device, Method, and User Interface for Processing Intensity of Touch Contacts

ABSTRACT

An electronic device includes a touch-sensitive surface. The electronic device includes one or more sensors to detect intensity of contacts with the touch-sensitive surface. The device detects a first touch input on the touch-sensitive surface, and, in response to detecting the first touch input on the touch-sensitive surface, determines a first intensity applied by the first touch input on the touch-sensitive surface. The device identifies a first intensity model identifier from a plurality of predefined intensity model identifiers, and, in accordance with the first intensity applied by the first touch input on the touch-sensitive surface and one or more thresholds associated with the first intensity model identifier, determines a first touch characterization parameter. Subsequent to determining the first touch characterization parameter, the device sends first touch information to the first software application. The first touch information includes the first intensity model identifier and the first touch characterization parameter.

RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 14/868,285, filed Sep. 28, 2015, entitled “Device,Method, and User Interface for Processing Intensity of Touch Contacts,”which claims priority to U.S. Provisional Patent Application Ser. No.62/129,937, filed Mar. 8, 2015, entitled “Device, Method, and UserInterface for Processing Intensity of Touch Contacts,” both of which areincorporated by reference herein in their entireties.

TECHNICAL FIELD

This relates generally to electronic devices with touch-sensitivesurfaces, including but not limited to electronic devices withtouch-sensitive surfaces and sensors to detect intensity of contacts onthe touch-sensitive surfaces.

BACKGROUND

The use of touch-sensitive surfaces as input devices for computers andother electronic computing devices has increased significantly in recentyears. Exemplary touch-sensitive surfaces include touchpads andtouch-screen displays. Such surfaces are widely used to manipulate userinterface objects on a display.

In addition to the presence or absence of a contacts (or touches) on thetouchpads and touch-screen displays, intensity of contacts can be usedto manipulate user interface objects on a display.

However, processing intensity of contacts can be cumbersome andinefficient. For example, processing intensity of contacts requirecomplex instructions, which can lead to increased computational load,increased size of software applications, and increased powerconsumption. These create a significant burden on the use of intensityof contacts in electronic devices.

SUMMARY

Accordingly, there is a need for electronic devices with faster, moreefficient methods and interfaces for processing touch inputs. Suchmethods and interfaces optionally complement or replace conventionalmethods for processing touch inputs. Such methods and interfaces reducethe number, extent, and/or nature of the inputs from a user and producea more efficient human-machine interface. For battery-operated devices,such methods and interfaces conserve power and increase the time betweenbattery charges.

The above deficiencies and other problems associated with userinterfaces for electronic devices with touch-sensitive surfaces arereduced or eliminated by the disclosed devices. In some embodiments, thedevice is a desktop computer. In some embodiments, the device isportable (e.g., a notebook computer, tablet computer, or handhelddevice). In some embodiments, the device is a personal electronic device(e.g., a wearable electronic device, such as a watch). In someembodiments, the device has a touchpad. In some embodiments, the devicehas a touch-sensitive display (also known as a “touch screen” or“touch-screen display”). In some embodiments, the device has a graphicaluser interface (GUI), one or more processors, memory and one or moremodules, programs or sets of instructions stored in the memory forperforming multiple functions. In some embodiments, the user interactswith the GUI primarily through stylus and/or finger contacts andgestures on the touch-sensitive surface. In some embodiments, thefunctions optionally include image editing, drawing, presenting, wordprocessing, spreadsheet making, game playing, telephoning, videoconferencing, e-mailing, instant messaging, workout support, digitalphotographing, digital videoing, web browsing, digital music playing,note taking, and/or digital video playing. Executable instructions forperforming these functions are, optionally, included in a non-transitorycomputer readable storage medium or other computer program productconfigured for execution by one or more processors.

In accordance with some embodiments, a method is performed at anelectronic device with a touch-sensitive surface and one or more sensorsto detect intensity of contacts with the touch-sensitive surface. Themethod includes: detecting a touch input on the touch-sensitive surface;in response to detecting the touch input on the touch-sensitive surface,determining an intensity of the touch input on the touch-sensitivesurface; and, in accordance with the intensity of the touch input on thetouch-sensitive surface and one or more preselected intensitythresholds, determining an intensity stage of the touch input. Theintensity stage of the touch input is selected from a plurality ofpredefined intensity stages. The method also includes processing thetouch input based on the intensity stage of the touch input.

In accordance with some embodiments, a method is performed at anelectronic device with a touch-sensitive surface and one or more sensorsto detect intensity of contacts with the touch-sensitive surface. Theelectronic device stores a first software application. The methodincludes: detecting a first touch input on the touch-sensitive surface;in response to detecting the first touch input on the touch-sensitivesurface, determining a first intensity applied by the first touch inputon the touch-sensitive surface; identifying a first intensity modelidentifier from a plurality of predefined intensity model identifiers;in accordance with the first intensity applied by the first touch inputon the touch-sensitive surface and one or more thresholds associatedwith the first intensity model identifier, determining a first touchcharacterization parameter; and, subsequent to determining the firsttouch characterization parameter, sending first touch information to thefirst software application. The first touch information includes thefirst intensity model identifier and the first touch characterizationparameter.

In accordance with some embodiments, a method is performed at anelectronic device with a touch-sensitive surface and one or more sensorsto detect intensity of contacts with the touch-sensitive surface. Theelectronic device stores a first software application. The methodincludes: detecting a first touch input on a first touch region of thetouch-sensitive surface; identifying a first intensity model identifier,associated with the first touch region of the touch-sensitive surface,from a plurality of predefined intensity model identifiers; in responseto detecting the first touch input on the first touch region of thetouch-sensitive surface: determining a first intensity applied by thefirst touch input on the first touch region of the touch-sensitivesurface; in accordance with the first intensity applied by the firsttouch input on the touch-sensitive surface and one or more thresholdsassociated with the first intensity model identifier, determining afirst touch characterization parameter; and, subsequent to determiningthe first touch characterization parameter, sending first touchinformation to the first software application. The first touchinformation includes the first touch characterization parameter. Themethod also includes detecting a second touch input on a second touchregion of the touch-sensitive surface. The second touch region of thetouch-sensitive surface is distinct from the first touch region of thetouch-sensitive surface. The method further includes identifying asecond intensity model identifier, associated with the second touchregion of the touch-sensitive surface, from the plurality of predefinedintensity model identifiers; and, in response to detecting the secondtouch input on the second touch region of the touch-sensitive surface:determining a second intensity applied by the second touch input on thesecond touch region of the touch-sensitive surface; in accordance withthe second intensity applied by the second touch input on thetouch-sensitive surface and one or more thresholds associated with thesecond intensity model identifier, determining a second touchcharacterization parameter; and, subsequent to determining the secondtouch characterization parameter, sending second touch information tothe first software application. The second touch information includesthe second touch characterization parameter.

In accordance with some embodiments, a method is performed at anelectronic device with a touch-sensitive surface and one or more sensorsto detect intensity of contacts with the touch-sensitive surface. Themethod includes: detecting a touch input on the touch-sensitive surface;and, in response to detecting the touch input: in accordance with adetermination that the touch input is at a location on thetouch-sensitive surface that is associated with a first intensity modelof a plurality of different intensity models, processing the touch inputin accordance with an intensity applied by the touch input on thetouch-sensitive surface and the first intensity model; and, inaccordance with a determination that the touch input is at a location onthe touch-sensitive surface that is associated with a second intensitymodel different from the first intensity model, processing the touchinput in accordance with an intensity applied by the touch input on thetouch-sensitive surface and the second intensity model.

In accordance with some embodiments, a method is performed at anelectronic device with a display, a touch-sensitive surface, and one ormore sensors to detect intensity of contacts with the touch-sensitivesurface. The electronic device stores a first software application. Themethod includes: displaying a user interface that includes two or moredisplay regions, including a first display region and a second displayregion; and, while displaying the user interface: detecting a firstintensity applied by a touch input at a first location on thetouch-sensitive surface that corresponds to the first display region;detecting a movement of the touch input across the touch-sensitivesurface from the first location on the touch-sensitive surface to asecond location on the touch-sensitive surface that corresponds to thesecond display region; after detecting the movement of the touch inputfrom the first location on the touch-sensitive surface to the secondlocation on the touch-sensitive surface, detecting a second intensityapplied by the touch input at the second location on the touch-sensitivesurface; and, in response to detecting the second intensity applied bythe touch input at the second location on the touch-sensitive surface:in accordance with a determination that the first intensity does notsatisfy a first intensity threshold, processing the second intensity inaccordance with one or more intensity thresholds associated with thesecond display region; and, in accordance with a determination that thefirst intensity satisfies the first intensity threshold, processing thesecond intensity in accordance with one or more intensity thresholdsassociated with the first display region.

In accordance with some embodiments, a method is performed at anelectronic device in communication with a plurality of input devicesincluding a first input device that is configured to generate tactileoutputs in response to inputs and a second input device that isconfigured to generate tactile outputs. The method includes receiving anindication of an input detected by a respective input device of theplurality of input devices, and in response to receiving the indicationof the input, providing information describing the input to anapplication running on the device that enables the application to reactto the input. The method further includes receiving a reaction to theinput from the application that indicates that a tactile output is to begenerated in response to the input, and in response to receiving thereaction to the input from the application, causing the generation of arespective tactile output. In accordance with a determination that thereaction was triggered by the first input device, the respective tactileoutput is generated at the first input device based on the reaction tothe input from the application, and in accordance with a determinationthat the reaction was triggered by the second input device, therespective tactile output is generated at the second input device basedon the reaction to the input from the application.

In accordance with some embodiments, an electronic device includes adisplay, a first input device that is configured to generate tactileoutputs in response to inputs, a second input device that is configuredto generate tactile outputs, one or more processors, memory, and one ormore programs, wherein the one or more programs are stored in the memoryand configured to be executed by the one or more processors. The one ormore programs including instructions for receiving an indication of aninput detected by a respective input device of the plurality of inputdevices, and in response to receiving the indication of the input,providing information describing the input to an application running onthe device that enables the application to react to the input. The oneor more programs further include instructions for receiving a reactionto the input from the application that indicates that a tactile outputis to be generated in response to the input, and in response toreceiving the reaction to the input from the application, causing thegeneration of a respective tactile output. In accordance with adetermination that the reaction was triggered by the first input device,the respective tactile output is generated at the first input devicebased on the reaction to the input from the application, and inaccordance with a determination that the reaction was triggered by thesecond input device, the respective tactile output is generated at thesecond input device based on the reaction to the input from theapplication.

In accordance with some embodiments, a non-transitory computer readablestorage medium stores one or more programs, the one or more programscomprising instructions, which when executed by an electronic devicehaving a display, a first input device that is configured to generatetactile outputs in response to inputs, and a second input device that isconfigured to generate tactile outputs, cause the device to perform theaforementioned method, or any of the other methods described herein.

In accordance with some embodiments, a method is performed at anelectronic device in communication with one or more input devices thatare configured to generate tactile outputs in response to inputs. Themethod includes receiving an indication of an input detected by arespective input device of the one or more input devices, and inresponse to receiving the indication of the input, providing informationdescribing the input to an application running on the device thatenables the application to react to the input. The method furtherincludes receiving a reaction to the input from the application thatindicates that a tactile output is to be generated in response to theinput. In response to receiving the reaction to the input from theapplication, the electronic device performs a set of operations,including comparing an input time for the reaction to an output time forthe reaction. With respect to the comparing, the input time for thereaction corresponds to a time at which the input was detected by therespective input device, and the output time for the reactioncorresponds to a time at which a tactile output corresponding to thereaction is configured to be generated at the respective input device.The set of operations performed in response to receiving the reaction tothe input from the application further includes determining whethertactile output criteria have been met, wherein the tactile outputcriteria include a criterion that is met when an input time is less thana predetermined amount of time before the output time, and in accordancewith a determination that the tactile output criteria have been met,causing generation, at the respective input device, of a tactile outputcorresponding to the reaction from the application. On the other hand,the set of operations performed in response to receiving the reaction tothe input from the application includes, in accordance with adetermination that the tactile output criteria have not been met,forgoing generation, at the respective input device, of the tactileoutput corresponding to the reaction from the application.

In accordance with some embodiments, an electronic device includes adisplay, one or more input devices that are configured to generatetactile outputs in response to inputs, one or more processors, memory,and one or more programs, wherein the one or more programs are stored inthe memory and configured to be executed by the one or more processors.The one or more programs including instructions for receiving anindication of an input detected by a respective input device of the oneor more input devices, and in response to receiving the indication ofthe input, providing information describing the input to an applicationrunning on the device that enables the application to react to theinput. The one or more programs further include instructions forreceiving a reaction to the input from the application that indicatesthat a tactile output is to be generated in response to the input, andin response to receiving the reaction to the input from the application,performing a set of operations, including comparing an input time forthe reaction to an output time for the reaction. With respect to thecomparing, the input time for the reaction corresponds to a time atwhich the input was detected by the respective input device, and theoutput time for the reaction corresponds to a time at which a tactileoutput corresponding to the reaction is configured to be generated atthe respective input device. The set of operations performed in responseto receiving the reaction to the input from the application furtherincludes determining whether tactile output criteria have been met,wherein the tactile output criteria include a criterion that is met whenan input time is less than a predetermined amount of time before theoutput time, and in accordance with a determination that the tactileoutput criteria have been met, causing generation, at the respectiveinput device, of a tactile output corresponding to the reaction from theapplication. On the other hand, the set of operations performed inresponse to receiving the reaction to the input from the applicationincludes, in accordance with a determination that the tactile outputcriteria have not been met, forgoing generation, at the respective inputdevice, of the tactile output corresponding to the reaction from theapplication.

In accordance with some embodiments, a non-transitory computer readablestorage medium stores one or more programs, the one or more programscomprising instructions, which when executed by an electronic devicehaving a display, one or more input devices that are configured togenerate tactile outputs in response to inputs, cause the device toperform the aforementioned method, or any of the other methods describedherein.

In accordance with some embodiments, an electronic device includes atouch-sensitive surface unit to receive contacts, one or more sensorunits to detect intensity of contacts with the touch-sensitive surfaceunit; and a processing unit coupled with the touch-sensitive surfaceunit and the one or more sensor units. The processing unit is configuredto: detect a touch input on the touch-sensitive surface unit; inresponse to detecting the touch input on the touch-sensitive surfaceunit, determine an intensity of the touch input on the touch-sensitivesurface unit; in accordance with the intensity of the touch input on thetouch-sensitive surface unit and one or more preselected intensitythresholds, determine an intensity stage of the touch input, wherein theintensity stage of the touch input is selected from a plurality ofpredefined intensity stages; and process the touch input based on theintensity stage of the touch input.

In accordance with some embodiments, an electronic device includes atouch-sensitive surface unit to receive contacts, one or more sensorunits to detect intensity of contacts with the touch-sensitive surfaceunit; and a processing unit coupled with the touch-sensitive surfaceunit and the one or more sensor units. The processing unit is configuredto: detect a first touch input on the touch-sensitive surface unit; inresponse to detecting the first touch input on the touch-sensitivesurface unit, determine a first intensity applied by the first touchinput on the touch-sensitive surface unit; identify a first intensitymodel identifier from a plurality of predefined intensity modelidentifiers; in accordance with the first intensity applied by the firsttouch input on the touch-sensitive surface unit and one or morethresholds associated with the first intensity model identifier,determine a first touch characterization parameter; and, subsequent todetermining the first touch characterization parameter, send first touchinformation to a first software application, wherein the first touchinformation includes the first intensity model identifier and the firsttouch characterization parameter.

In accordance with some embodiments, an electronic device includes adisplay unit configured to display a touch-sensitive surface unit toreceive contacts, one or more sensor units to detect intensity ofcontacts with the touch-sensitive surface unit; and a processing unitcoupled with the touch-sensitive surface unit and the one or more sensorunits. The processing unit is configured to: detect a first touch inputon a first touch region of the touch-sensitive surface unit; identify afirst intensity model identifier, associated with the first touch regionof the touch-sensitive surface unit, from a plurality of predefinedintensity model identifiers; and, in response to detecting the firsttouch input on the first touch region of the touch-sensitive surfaceunit: determine a first intensity of the first touch input on the firsttouch region of the touch-sensitive surface unit; in accordance with thefirst intensity applied by the first touch input on the touch-sensitivesurface unit and one or more thresholds associated with the firstintensity model identifier, determine a first touch characterizationparameter; and, subsequent to determining the first touchcharacterization parameter, send first touch information to a firstsoftware application, wherein the first touch information includes thefirst touch characterization parameter. The processing unit is alsoconfigured to: detect a second touch input on a second touch region ofthe touch-sensitive surface unit, wherein the second touch region of thetouch-sensitive surface unit is distinct from the first touch region ofthe touch-sensitive surface unit; identify a second intensity modelidentifier, associated with the second touch region of thetouch-sensitive surface unit, from the plurality of predefined intensitymodel identifiers; and, in response to detecting the second touch inputon the second touch region of the touch-sensitive surface unit:determine a second intensity applied by the second touch input on thesecond touch region of the touch-sensitive surface unit; in accordancewith the second intensity applied by the second touch input on thetouch-sensitive surface unit and one or more thresholds associated withthe second intensity model identifier, determine a second touchcharacterization parameter; and, subsequent to determining the secondtouch characterization parameter, send second touch information to thefirst software application, wherein the second touch informationincludes the second touch characterization parameter.

In accordance with some embodiments, an electronic device includes atouch-sensitive surface unit to receive contacts, one or more sensorunits to detect intensity of contacts with the touch-sensitive surfaceunit; and a processing unit coupled with the touch-sensitive surfaceunit and the one or more sensor units. The processing unit is configuredto: detect a touch input on the touch-sensitive surface unit; and, inresponse to detecting the touch input: in accordance with adetermination that the touch input is at a location on thetouch-sensitive surface unit that is associated with a first intensitymodel of a plurality of different intensity models, processing the touchinput in accordance with an intensity applied by the touch input on thetouch-sensitive surface unit and the first intensity model; and, inaccordance with a determination that the touch input is at a location onthe touch-sensitive surface unit that is associated with a secondintensity model different from the first intensity model, processing thetouch input in accordance with an intensity applied by the touch inputon the touch-sensitive surface unit and the second intensity model.

In accordance with some embodiments, an electronic device includes adisplay unit configured to display a user interface, a touch-sensitivesurface unit to receive contacts, one or more sensor units to detectintensity of contacts with the touch-sensitive surface unit; and aprocessing unit coupled with the display unit, the touch-sensitivesurface unit, and the one or more sensor units. The processing unit isconfigured to: enable display of a user interface that includes two ormore display regions, including a first display region and a seconddisplay region on the display unit; and, while displaying the userinterface: detect a first intensity applied by a touch input at a firstlocation on the touch-sensitive surface unit that corresponds to thefirst display region; detect a movement of the touch input across thetouch-sensitive surface unit from the first location on thetouch-sensitive surface unit to a second location on the touch-sensitivesurface unit that corresponds to the second display region; afterdetecting the movement of the touch input from the first location on thetouch-sensitive surface unit to the second location on thetouch-sensitive surface unit, detect a second intensity applied by thetouch input at the second location on the touch-sensitive surface unit;and, in response to detecting the second intensity applied by the touchinput at the second location on the touch-sensitive surface unit: inaccordance with a determination that the first intensity does notsatisfy a first intensity threshold, process the second intensity inaccordance with one or more intensity thresholds associated with thesecond display region; and, in accordance with a determination that thefirst intensity satisfies the first intensity threshold, process thesecond intensity in accordance with one or more intensity thresholdsassociated with the first display region.

In accordance with some embodiments, an electronic device includes adisplay unit configured to display a user interface and a plurality ofinput units, including a first input unit that is configured to generatetactile outputs in response to inputs, and a second input unit that isconfigured to generate tactile outputs. The electronic device furtherincludes an input indication receiving unit configured to receive anindication of an input detected by a respective input unit of theplurality of input units, and an information providing unit configuredto provide, in response to receiving the indication of the input,information describing the input to an application running on theelectronic device that enables the application to react to the input.The electronic device further includes a reaction receiving unitconfigured to receive a reaction to the input from the application thatindicates that a tactile output is to be generated in response to theinput, and a causing unit configured to cause, in response to receivingthe reaction to the input from the application, the generation of arespective tactile output. In accordance with a determination that thereaction was triggered by the first input unit, the respective tactileoutput is generated at the first input unit based on the reaction to theinput from the application, and in accordance with a determination thatthe reaction was triggered by the second input unit, the respectivetactile output is generated at the second input unit based on thereaction to the input from the application.

In accordance with some embodiments, an electronic device includes adisplay unit configured to display a user interface hierarchy and one ormore input units that are configured to generate tactile outputs inresponse to inputs. The electronic device further includes an inputindication receiving unit configured to receive an indication of aninput detected by a respective input unit of the one or more inputunits, and an information providing unit configured to provide, inresponse to receiving the indication of the input, informationdescribing the input to an application running on the electronic devicethat enables the application to react to the input. The electronicdevice further includes a reaction receiving unit configured to receivea reaction to the input from the application that indicates that atactile output is to be generated in response to the input, and acausing unit configured to cause, in response to receiving the reactionto the input from the application, the performance of a set ofoperations, including comparing an input time for the reaction to anoutput time for the reaction. With respect to the comparing, the inputtime for the reaction corresponds to a time at which the input wasdetected by the respective input device, and the output time for thereaction corresponds to a time at which a tactile output correspondingto the reaction is configured to be generated at the respective inputdevice. The set of operations performed in response to receiving thereaction to the input from the application further includes determiningwhether tactile output criteria have been met, wherein the tactileoutput criteria include a criterion that is met when an input time isless than a predetermined amount of time before the output time, and inaccordance with a determination that the tactile output criteria havebeen met, causing generation, at the respective input device, of atactile output corresponding to the reaction from the application. Onthe other hand, the set of operations performed in response to receivingthe reaction to the input from the application includes, in accordancewith a determination that the tactile output criteria have not been met,forgoing generation, at the respective input device, of the tactileoutput corresponding to the reaction from the application.

In accordance with some embodiments, an electronic device includes atouch-sensitive surface, one or more sensors to detect intensity ofcontacts with the touch-sensitive surface, optionally a display, one ormore processors, and memory. The memory stores one or more programs thatare configured to be executed by the one or more processors. The one ormore programs include instructions for performing or causing performanceof the operations of any of the methods described herein. In accordancewith some embodiments, a computer readable storage medium stores thereininstructions, which, when executed by an electronic device with atouch-sensitive surface and one or more sensors to detect intensity ofcontacts with the touch-sensitive surface, cause the device to performor cause performance of the operations of any of the methods describedherein. In accordance with some embodiments, a graphical user interfaceon an electronic device with a display, a touch-sensitive surface, oneor more sensors to detect intensity of contacts with the touch-sensitivesurface, a memory, and one or more processors to execute one or moreprograms stored in the memory includes one or more of the elementsdisplayed in any of the methods described herein, which are updated inresponse to inputs, as described in any of the methods described herein.In accordance with some embodiments, an electronic device includes: atouch-sensitive surface, one or more sensors to detect intensity ofcontacts with the touch-sensitive surface, optionally a display, andmeans for performing or causing performance of the operations of any ofthe methods described herein. In accordance with some embodiments, aninformation processing apparatus, for use in an electronic device with atouch-sensitive surface, one or more sensors to detect intensity ofcontacts with the touch-sensitive surface, and optionally a displayincludes means for performing or causing performance of the operationsof any of the methods described herein.

In accordance with some embodiments, at least one of the aforementionedinput devices has a touch-sensitive surface and one or more sensors todetect intensity of contacts with the touch-sensitive surface.Furthermore, in some embodiments, any of the aforementioned methodsincludes displaying, on the display, a user interface for the electronicdevice and detecting an input (or sequence of inputs) on thetouch-sensitive surface of one of the input devices. Optionally, any ofthe aforementioned methods includes detecting changes in characteristicsof a respective contact that is continuously detected on thetouch-sensitive surface.

Thus, electronic devices with displays, one or more input devices thatare configured to generate tactile outputs in response to inputs (e.g.,including one or more input devices having touch-sensitive surfaces, andone or more sensors to detect intensity of contacts with thetouch-sensitive surface) are provided with faster, more efficientmethods and interfaces for performing an operation in accordance with aselected mode of operation, thereby increasing the effectiveness,efficiency, and user satisfaction with such devices. Such methods andinterfaces may complement or replace conventional methods for performingan operation in accordance with a selected mode of operation.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the various described embodiments,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1A is a block diagram illustrating a portable multifunction devicewith a touch-sensitive display in accordance with some embodiments.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments.

FIG. 2 illustrates a portable multifunction device having a touch screenin accordance with some embodiments.

FIG. 3A is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments.

FIG. 3B is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments.

FIG. 3C is a perspective view of an exemplary multifunction device witha touch-sensitive surface in accordance with some embodiments.

FIG. 3D is a perspective view of an input device incorporating atouch-sensitive surface separate from a multifunction device inaccordance with some embodiments.

FIG. 3E is a block diagram of an exemplary multifunction device thatincludes a trackpad in accordance with some embodiments.

FIG. 3F is a simplified block diagram illustrating architecture of anexemplary multifunction device in accordance with some embodiments.

FIG. 3G is a block diagram illustrating data structures used by anexemplary multifunction device in accordance with some embodiments.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on a portable multifunction device in accordance with someembodiments.

FIG. 4B illustrates an exemplary user interface for a multifunctiondevice with a touch-sensitive surface that is separate from the displayin accordance with some embodiments.

FIGS. 4C-4E illustrate exemplary dynamic intensity thresholds inaccordance with some embodiments.

FIGS. 5A-5TT illustrate exemplary user interfaces for processing touchinputs in accordance with some embodiments.

FIGS. 6A-6C are flow diagrams illustrating a method of processing atouch input based on an intensity stage of the touch input in accordancewith some embodiments.

FIG. 6D is a flow diagram illustrating a method of processing touchinformation that includes an intensity stage of a touch input inaccordance with some embodiments.

FIGS. 7A-7C are flow diagrams illustrating a method of processing touchinputs based on intensity model identifiers in accordance with someembodiments.

FIG. 7D is a flow diagram illustrating a method of processing touchinformation based on an intensity model identifier in accordance withsome embodiments.

FIGS. 8A-8C are flow diagrams illustrating a method of processing touchinputs in different regions based on distinct intensity models inaccordance with some embodiments.

FIGS. 9A-9D are flow diagrams illustrating a method of processing atouch input based on a location-related intensity model in accordancewith some embodiments.

FIGS. 10A-10D are flow diagrams illustrating a method of processing atouch input based on latching of the touch input in accordance with someembodiments.

FIGS. 11A-11C are flow diagrams illustrating a method of routing atactile output to an input device of a plurality of input devices inaccordance with some embodiments.

FIGS. 12A-12C are flow diagrams illustrating a method of conditionallycanceling or forgoing generation of a tactile output at a respectiveinput device in accordance with some embodiments.

FIGS. 13-19 are functional block diagrams of an electronic device inaccordance with some embodiments.

FIG. 20A illustrates stage zones for an exemplary gesture in accordancewith some embodiments.

FIGS. 20B-20E illustrate exemplary intensity models in accordance withsome embodiments.

DESCRIPTION OF EMBODIMENTS

Many electronic devices have user interfaces in which multipleoperations are, optionally, performed in response to gestures performedusing one or more input devices. In many contexts, in addition toproviding the user with visual feedback on a display, to let the userknow how the electronic device has responded to those gestures, it wouldbe beneficial to provide a tactile output or tactile feedback. Forexample, such tactile feedback could inform the user as to when agesture has meet predefined criteria for performing a particularoperation, and/or could inform the user as to when a gesture hasviolated predefined criteria (e.g., for performing a particularoperation or more generally for using a particular application ormanipulating a user interface). The embodiments described below improveon methods for provide tactile feedback in response to an input receiveddetected by a respective input device. For example, some embodimentsdescribed below, have one or more components, separate from anapplication that responds to information describing an input, forprocessing and routing inputs to the application, for receiving areaction to the input from the application that indicates that a tactileoutput is to be generated in response to the input, and for thendetermining how to implement or otherwise handle the reaction receivedfrom the application.

In accordance with some embodiments, in response to detecting a touchinput, the electronic device processes a touch input based on anintensity stage of the touch input. More specifically, the devicedetermines the intensity stage of the touch input (e.g., at a contactintensity module) so that a software application that is separate fromthe contact intensity module does not need to determine the intensitystage of the touch input on its own.

In accordance with some embodiments, in response to detecting a touchinput, the electronic device determines a touch characterizationparameter for the touch input (e.g., at a contact intensity module) andsends the touch characterization parameter to a software applicationthat is separate from the contact intensity module so that the softwareapplication does not need to directly process raw data for the touchinput. More specifically, the electronic device uses an intensity modelto determine the touch characterization parameter.

In accordance with some embodiments in which intensity models are mappedto different regions of a touch-sensitive surface, in response to atouch input, the electronic device uses an intensity model thatcorresponds to a location of the touch input to determine a touchcharacterization parameter. More specifically, different intensitymodels are used for different regions.

In accordance with some embodiments, in response to detecting movementof a touch input, the electronic device determines whether the touchinput has latched to a particular region of the user interface. If thetouch input is latched, the electronic device processes the touch inputbased on an intensity model associated with the latched region. If thetouch input is not latched, the electronic device processes the touchinput based on an intensity model associated with a current position ofa touch input (or a cursor).

In accordance with some embodiments in which the electronic device is incommunication with a plurality of input devices, including a first inputdevice that is configured to generate tactile outputs in response toinputs and a second input device that is configured to generate tactileoutputs, in response to receiving the reaction to the input from theapplication, the device causes the generation of a respective tactileoutput at the same input device as the input device from which an inputwas received that triggered the reaction. More specifically, inaccordance with a determination that the reaction was triggered by thefirst input device, the respective tactile output is generated at thefirst input device based on the reaction to the input from theapplication, and in accordance with a determination that the reactionwas triggered by the second input device, the respective tactile outputis generated at the second input device based on the reaction to theinput from the application.

In accordance with some embodiments in which the electronic device is incommunication with one or more input devices that are configured togenerate tactile outputs in response to inputs, in response to receivingthe reaction to the input from the application, the device performs aset of operations. The set of operations include operations thatconditionally cancel or forgo generation of a tactile outputcorresponding to the reaction from the application in accordance with adetermination that tactile output criteria have not been met. Thetactile output criteria include a criterion that is met when an inputtime is less than a predetermined amount of time before the output time.

Below, FIGS. 1A-1B, 2, and 3A-3G illustrate exemplary devices. FIGS.4A-4B and 5A-5TT illustrate exemplary user interfaces for processingtouch inputs. FIGS. 6A-6C illustrate a flow diagram of a method ofprocessing a touch input based an intensity stage of the touch input inaccordance with some embodiments. FIG. 6D is a flow diagram illustratinga method of processing touch information that includes an intensitystage of a touch input in accordance with some embodiments. FIGS. 7A-7Cillustrate a flow diagram of a method of processing touch inputs basedon intensity model identifiers in accordance with some embodiments. FIG.7D is a flow diagram illustrating a method of processing touchinformation based on an intensity model identifier in accordance withsome embodiments. FIGS. 8A-8C illustrate a flow diagram of a method ofprocessing touch inputs in different regions based on distinct intensitymodels in accordance with some embodiments. FIGS. 9A-9D illustrate aflow diagram of a method of processing a touch input based on anintensity model associated with a location of the touch input inaccordance with some embodiments. FIGS. 10A-10D illustrate a flowdiagram of a method of processing a touch input based on whether thetouch input has latched on to a particular region in accordance withsome embodiments. FIGS. 11A-11C are flow diagrams illustrating a methodof routing a tactile output to an input device of a plurality of inputdevices in accordance with some embodiments. FIGS. 12A-12C are flowdiagrams illustrating a method of conditionally canceling or forgoinggeneration of a tactile output at a respective input device inaccordance with some embodiments. The user interfaces in FIGS. 5A-5TTare used to illustrate the processes in FIGS. 6A-6D, 7A-7D, 8A-8C,9A-9D, 10A-10D, 11A-11C, and 12A-12C.

Exemplary Devices

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the various described embodiments. However,it will be apparent to one of ordinary skill in the art that the variousdescribed embodiments may be practiced without these specific details.In other instances, well-known methods, procedures, components,circuits, and networks have not been described in detail so as not tounnecessarily obscure aspects of the embodiments.

It will also be understood that, although the terms first, second, etc.are, in some instances, used herein to describe various elements, theseelements should not be limited by these terms. These terms are only usedto distinguish one element from another. For example, a first contactcould be termed a second contact, and, similarly, a second contact couldbe termed a first contact, without departing from the scope of thevarious described embodiments. The first contact and the second contactare both contacts, but they are not the same contact, unless the contextclearly indicates otherwise.

The terminology used in the description of the various describedembodiments herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used in thedescription of the various described embodiments and the appendedclaims, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

As used herein, the term “if” is, optionally, construed to mean “when”or “upon” or “in response to determining” or “in response to detecting,”depending on the context. Similarly, the phrase “if it is determined” or“if [a stated condition or event] is detected” is, optionally, construedto mean “upon determining” or “in response to determining” or “upondetecting [the stated condition or event]” or “in response to detecting[the stated condition or event],” depending on the context.

Embodiments of electronic devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the device is a portable communications device, such as amobile telephone, that also contains other functions, such as PDA and/ormusic player functions. Exemplary embodiments of portable multifunctiondevices include, without limitation, the iPhone®, iPod Touch®, and iPad®devices from Apple Inc. of Cupertino, Calif. Other portable electronicdevices, such as laptops or tablet computers with touch-sensitivesurfaces (e.g., touch-screen displays and/or touchpads), are,optionally, used. It should also be understood that, in someembodiments, the device is not a portable communications device, but isa desktop computer with a touch-sensitive surface (e.g., a touch-screendisplay and/or a touchpad).

In the discussion that follows, an electronic device that includes adisplay and a touch-sensitive surface is described. It should beunderstood, however, that the electronic device optionally includes oneor more other physical user-interface devices, such as a physicalkeyboard, a mouse and/or a joystick.

The device typically supports a variety of applications, such as one ormore of the following: a note taking application, a drawing application,a presentation application, a word processing application, a websitecreation application, a disk authoring application, a spreadsheetapplication, a gaming application, a telephone application, a videoconferencing application, an e-mail application, an instant messagingapplication, a workout support application, a photo managementapplication, a digital camera application, a digital video cameraapplication, a web browsing application, a digital music playerapplication, and/or a digital video player application.

The various applications that are executed on the device optionally useat least one common physical user-interface device, such as thetouch-sensitive surface. One or more functions of the touch-sensitivesurface as well as corresponding information displayed on the deviceare, optionally, adjusted and/or varied from one application to the nextand/or within a respective application. In this way, a common physicalarchitecture (such as the touch-sensitive surface) of the deviceoptionally supports the variety of applications with user interfacesthat are intuitive and transparent to the user.

Attention is now directed toward embodiments of portable devices withtouch-sensitive displays. FIG. 1A is a block diagram illustratingportable multifunction device 100 with touch-sensitive display system112 in accordance with some embodiments. Touch-sensitive display system112 is sometimes called a “touch screen” for convenience, and issometimes simply called a touch-sensitive display. Device 100 includesmemory 102 (which optionally includes one or more computer readablestorage mediums), memory controller 122, one or more processing units(CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry110, speaker 111, microphone 113, input/output (I/O) subsystem 106,other input or control devices 116, and external port 124. Device 100optionally includes one or more optical sensors 164. Device 100optionally includes one or more intensity sensors 165 for detectingintensity of contacts on device 100 (e.g., a touch-sensitive surfacesuch as touch-sensitive display system 112 of device 100). Device 100optionally includes one or more tactile output generators 167 forgenerating tactile outputs on device 100 (e.g., generating tactileoutputs on a touch-sensitive surface such as touch-sensitive displaysystem 112 of device 100 or touchpad 355 of device 300). Thesecomponents optionally communicate over one or more communication busesor signal lines 103.

As used in the specification and claims, the term “tactile output”refers to physical displacement of a device relative to a previousposition of the device, physical displacement of a component (e.g., atouch-sensitive surface) of a device relative to another component(e.g., housing) of the device, or displacement of the component relativeto a center of mass of the device that will be detected by a user withthe user's sense of touch. For example, in situations where the deviceor the component of the device is in contact with a surface of a userthat is sensitive to touch (e.g., a finger, palm, or other part of auser's hand), the tactile output generated by the physical displacementwill be interpreted by the user as a tactile sensation corresponding toa perceived change in physical characteristics of the device or thecomponent of the device. For example, movement of a touch-sensitivesurface (e.g., in a touch-sensitive display or trackpad) is, optionally,interpreted by the user as a “down click” or “up click” of a physicalactuator button. In some cases, a user will feel a tactile sensationsuch as an “down click” or “up click” even when there is no movement ofa physical actuator button associated with the touch-sensitive surfacethat is physically pressed (e.g., displaced) by the user's movements. Asanother example, movement of the touch-sensitive surface is, optionally,interpreted or sensed by the user as “roughness” of the touch-sensitivesurface, even when there is no change in smoothness of thetouch-sensitive surface. While such interpretations of touch by a userwill be subject to the individualized sensory perceptions of the user,there are many sensory perceptions of touch that are common to a largemajority of users. Thus, when a tactile output is described ascorresponding to a particular sensory perception of a user (e.g., an “upclick,” a “down click,” “roughness”), unless otherwise stated, thegenerated tactile output corresponds to physical displacement of thedevice or a component thereof that will generate the described sensoryperception for a typical (or average) user.

It should be appreciated that device 100 is only one example of aportable multifunction device, and that device 100 optionally has moreor fewer components than shown, optionally combines two or morecomponents, or optionally has a different configuration or arrangementof the components. The various components shown in FIG. 1A areimplemented in hardware, software, firmware, or a combination thereof,including one or more signal processing and/or application specificintegrated circuits.

Memory 102 optionally includes high-speed random access memory andoptionally also includes non-volatile memory, such as one or moremagnetic disk storage devices, flash memory devices, or othernon-volatile solid-state memory devices. Access to memory 102 by othercomponents of device 100, such as CPU(s) 120 and the peripheralsinterface 118, is, optionally, controlled by memory controller 122.

Peripherals interface 118 can be used to couple input and outputperipherals of the device to CPU(s) 120 and memory 102. The one or moreprocessors 120 run or execute various software programs and/or sets ofinstructions stored in memory 102 to perform various functions fordevice 100 and to process data.

In some embodiments, peripherals interface 118, CPU(s) 120, and memorycontroller 122 are, optionally, implemented on a single chip, such aschip 104. In some other embodiments, they are, optionally, implementedon separate chips.

RF (radio frequency) circuitry 108 receives and sends RF signals, alsocalled electromagnetic signals. RF circuitry 108 converts electricalsignals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. RF circuitry 108 optionally includes well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. RFcircuitry 108 optionally communicates with networks, such as theInternet, also referred to as the World Wide Web (WWW), an intranetand/or a wireless network, such as a cellular telephone network, awireless local area network (LAN) and/or a metropolitan area network(MAN), and other devices by wireless communication. The wirelesscommunication optionally uses any of a plurality of communicationsstandards, protocols and technologies, including but not limited toGlobal System for Mobile Communications (GSM), Enhanced Data GSMEnvironment (EDGE), high-speed downlink packet access (HSDPA),high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO),HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), nearfield communication (NFC), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a,IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and/or IEEE802.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocol fore-mail (e.g., Internet message access protocol (IMAP) and/or post officeprotocol (POP)), instant messaging (e.g., extensible messaging andpresence protocol (XMPP), Session Initiation Protocol for InstantMessaging and Presence Leveraging Extensions (SIMPLE), Instant Messagingand Presence Service (IMPS)), and/or Short Message Service (SMS), or anyother suitable communication protocol, including communication protocolsnot yet developed as of the filing date of this document.

Audio circuitry 110, speaker 111, and microphone 113 provide an audiointerface between a user and device 100. Audio circuitry 110 receivesaudio data from peripherals interface 118, converts the audio data to anelectrical signal, and transmits the electrical signal to speaker 111.Speaker 111 converts the electrical signal to human-audible sound waves.Audio circuitry 110 also receives electrical signals converted bymicrophone 113 from sound waves. Audio circuitry 110 converts theelectrical signal to audio data and transmits the audio data toperipherals interface 118 for processing. Audio data is, optionally,retrieved from and/or transmitted to memory 102 and/or RF circuitry 108by peripherals interface 118. In some embodiments, audio circuitry 110also includes a headset jack (e.g., 212, FIG. 2). The headset jackprovides an interface between audio circuitry 110 and removable audioinput/output peripherals, such as output-only headphones or a headsetwith both output (e.g., a headphone for one or both ears) and input(e.g., a microphone).

I/O subsystem 106 couples input/output peripherals on device 100, suchas touch-sensitive display system 112 and other input or control devices116, with peripherals interface 118. I/O subsystem 106 optionallyincludes display controller 156, optical sensor controller 158,intensity sensor controller 159, haptic feedback controller 161, and oneor more input controllers 160 for other input or control devices. Theone or more input controllers 160 receive/send electrical signalsfrom/to other input or control devices 116. The other input or controldevices 116 optionally include physical buttons (e.g., push buttons,rocker buttons, etc.), dials, slider switches, joysticks, click wheels,and so forth. In some alternate embodiments, input controller(s) 160are, optionally, coupled with any (or none) of the following: akeyboard, infrared port, USB port, stylus, and/or a pointer device suchas a mouse. The one or more buttons (e.g., 208, FIG. 2) optionallyinclude an up/down button for volume control of speaker 111 and/ormicrophone 113. The one or more buttons optionally include a push button(e.g., 206, FIG. 2).

Touch-sensitive display system 112 provides an input interface and anoutput interface between the device and a user. Display controller 156receives and/or sends electrical signals from/to touch-sensitive displaysystem 112. Touch-sensitive display system 112 displays visual output tothe user. The visual output optionally includes graphics, text, icons,video, and any combination thereof (collectively termed “graphics”). Insome embodiments, some or all of the visual output corresponds touser-interface objects.

Touch-sensitive display system 112 has a touch-sensitive surface, sensoror set of sensors, that accepts input from the user based on hapticand/or tactile contact. Touch-sensitive display system 112 and displaycontroller 156 (along with any associated modules and/or sets ofinstructions in memory 102) detect contact (and any movement or breakingof the contact) on touch-sensitive display system 112 and converts thedetected contact into interaction with user-interface objects (e.g., oneor more soft keys, icons, web pages or images) that are displayed ontouch-sensitive display system 112. In some exemplary embodiments, apoint of contact between touch-sensitive display system 112 and the usercorresponds to a finger of the user or a stylus.

Touch-sensitive display system 112 optionally uses LCD (liquid crystaldisplay) technology, LPD (light emitting polymer display) technology, orLED (light emitting diode) technology, although other displaytechnologies are used in other embodiments. Touch-sensitive displaysystem 112 and display controller 156 optionally detect contact and anymovement or breaking thereof using any of a plurality of touch sensingtechnologies now known or later developed, including but not limited tocapacitive, resistive, infrared, and surface acoustic wave technologies,as well as other proximity sensor arrays or other elements fordetermining one or more points of contact with touch-sensitive displaysystem 112. In some exemplary embodiments, projected mutual capacitancesensing technology is used, such as that found in the iPhone®, iPodTouch®, and iPad® from Apple Inc. of Cupertino, Calif.

Touch-sensitive display system 112 optionally has a video resolution inexcess of 100 dpi. In some embodiments, the touch screen videoresolution is in excess of 400 dpi (e.g., 500 dpi, 800 dpi, or greater).The user optionally makes contact with touch-sensitive display system112 using any suitable object or appendage, such as a stylus, a finger,and so forth. In some embodiments, the user interface is designed towork with finger-based contacts and gestures, which can be less precisethan stylus-based input due to the larger area of contact of a finger onthe touch screen. In some embodiments, the device translates the roughfinger-based input into a precise pointer/cursor position or command forperforming the actions desired by the user.

In some embodiments, in addition to the touch screen, device 100optionally includes a touchpad (not shown) for activating ordeactivating particular functions. In some embodiments, the touchpad isa touch-sensitive area of the device that, unlike the touch screen, doesnot display visual output. The touchpad is, optionally, atouch-sensitive surface that is separate from touch-sensitive displaysystem 112 or an extension of the touch-sensitive surface formed by thetouch screen.

Device 100 also includes power system 162 for powering the variouscomponents. Power system 162 optionally includes a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light-emittingdiode (LED)) and any other components associated with the generation,management and distribution of power in portable devices.

Device 100 optionally also includes one or more optical sensors 164.FIG. 1A shows an optical sensor coupled with optical sensor controller158 in I/O subsystem 106. Optical sensor(s) 164 optionally includecharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) phototransistors. Optical sensor(s) 164 receive light from theenvironment, projected through one or more lens, and converts the lightto data representing an image. In conjunction with imaging module 143(also called a camera module), optical sensor(s) 164 optionally capturestill images and/or video. In some embodiments, an optical sensor islocated on the back of device 100, opposite touch-sensitive displaysystem 112 on the front of the device, so that the touch screen isenabled for use as a viewfinder for still and/or video imageacquisition. In some embodiments, another optical sensor is located onthe front of the device so that the user's image is obtained (e.g., forselfies, for videoconferencing while the user views the other videoconference participants on the touch screen, etc.).

Device 100 optionally also includes one or more contact intensitysensors 165. FIG. 1A shows a contact intensity sensor coupled withintensity sensor controller 159 in I/O subsystem 106. Contact intensitysensor(s) 165 optionally include one or more piezoresistive straingauges, capacitive force sensors, electric force sensors, piezoelectricforce sensors, optical force sensors, capacitive touch-sensitivesurfaces, or other intensity sensors (e.g., sensors used to measure theforce (or pressure) of a contact on a touch-sensitive surface). Contactintensity sensor(s) 165 receive contact intensity information (e.g.,pressure information or a proxy for pressure information) from theenvironment. In some embodiments, at least one contact intensity sensoris collocated with, or proximate to, a touch-sensitive surface (e.g.,touch-sensitive display system 112). In some embodiments, at least onecontact intensity sensor is located on the back of device 100, oppositetouch-screen display system 112 which is located on the front of device100.

Device 100 optionally also includes one or more proximity sensors 166.FIG. 1A shows proximity sensor 166 coupled with peripherals interface118. Alternately, proximity sensor 166 is coupled with input controller160 in I/O subsystem 106. In some embodiments, the proximity sensorturns off and disables touch-sensitive display system 112 when themultifunction device is placed near the user's ear (e.g., when the useris making a phone call).

Device 100 optionally also includes one or more tactile outputgenerators 167. FIG. 1A shows a tactile output generator coupled withhaptic feedback controller 161 in I/O subsystem 106. Tactile outputgenerator(s) 167 optionally include one or more electroacoustic devicessuch as speakers or other audio components and/or electromechanicaldevices that convert energy into linear motion such as a motor,solenoid, electroactive polymer, piezoelectric actuator, electrostaticactuator, or other tactile output generating component (e.g., acomponent that converts electrical signals into tactile outputs on thedevice). In some embodiments, tactile output generator(s) 167 receivetactile feedback generation instructions from haptic feedback module 133and generates tactile outputs on device 100 that are capable of beingsensed by a user of device 100. In some embodiments, at least onetactile output generator is collocated with, or proximate to, atouch-sensitive surface (e.g., touch-sensitive display system 112) and,optionally, generates a tactile output by moving the touch-sensitivesurface vertically (e.g., in/out of a surface of device 100) orlaterally (e.g., back and forth in the same plane as a surface of device100). In some embodiments, at least one tactile output generator sensoris located on the back of device 100, opposite touch-sensitive displaysystem 112, which is located on the front of device 100.

Device 100 optionally also includes one or more accelerometers 168. FIG.1A shows accelerometer 168 coupled with peripherals interface 118.Alternately, accelerometer 168 is, optionally, coupled with an inputcontroller 160 in I/O subsystem 106. In some embodiments, information isdisplayed on the touch-screen display in a portrait view or a landscapeview based on an analysis of data received from the one or moreaccelerometers. Device 100 optionally includes, in addition toaccelerometer(s) 168, a magnetometer (not shown) and a GPS (or GLONASSor other global navigation system) receiver (not shown) for obtaininginformation concerning the location and orientation (e.g., portrait orlandscape) of device 100.

In some embodiments, the software components stored in memory 102include operating system 126, communication module (or set ofinstructions) 128, contact/motion module (or set of instructions) 130,graphics module (or set of instructions) 132, haptic feedback module (orset of instructions) 133, text input module (or set of instructions)134, Global Positioning System (GPS) module (or set of instructions)135, and applications (or sets of instructions) 136. Furthermore, insome embodiments, memory 102 stores device/global internal state 157, asshown in FIGS. 1A and 3A-3B. Device/global internal state 157 includesone or more of: active application state, indicating which applications,if any, are currently active; display state, indicating whatapplications, views or other information occupy various regions oftouch-sensitive display system 112; sensor state, including informationobtained from the device's various sensors and other input or controldevices 116; and location and/or positional information concerning thedevice's location and/or attitude.

Operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X,WINDOWS, or an embedded operating system such as VxWorks) includesvarious software components and/or drivers for controlling and managinggeneral system tasks (e.g., memory management, storage device control,power management, etc.) and facilitates communication between varioushardware and software components.

Communication module 128 facilitates communication with other devicesover one or more external ports 124 and also includes various softwarecomponents for handling data received by RF circuitry 108 and/orexternal port 124. External port 124 (e.g., Universal Serial Bus (USB),FIREWIRE, etc.) is adapted for coupling directly to other devices orindirectly over a network (e.g., the Internet, wireless LAN, etc.). Insome embodiments, the external port is a multi-pin (e.g., 30-pin)connector that is the same as, or similar to and/or compatible with the30-pin connector used in some iPhone®, iPod Touch®, and iPad® devicesfrom Apple Inc. of Cupertino, Calif. In some embodiments, the externalport is a Lightning connector that is the same as, or similar to and/orcompatible with the Lightning connector used in some iPhone®, iPodTouch®, and iPad® devices from Apple Inc. of Cupertino, Calif.

Contact/motion module 130 optionally detects contact withtouch-sensitive display system 112 (in conjunction with displaycontroller 156) and other touch-sensitive devices (e.g., a touchpad orphysical click wheel). Contact/motion module 130 includes varioussoftware components for performing various operations related todetection of contact (e.g., by a finger or by a stylus), such asdetermining if contact has occurred (e.g., detecting a finger-downevent), determining an intensity of the contact (e.g., the force orpressure of the contact or a substitute for the force or pressure of thecontact), determining if there is movement of the contact and trackingthe movement across the touch-sensitive surface (e.g., detecting one ormore finger-dragging events), and determining if the contact has ceased(e.g., detecting a finger-up event or a break in contact).Contact/motion module 130 receives contact data from the touch-sensitivesurface. Determining movement of the point of contact, which isrepresented by a series of contact data, optionally includes determiningspeed (magnitude), velocity (magnitude and direction), and/or anacceleration (a change in magnitude and/or direction) of the point ofcontact. These operations are, optionally, applied to single contacts(e.g., one finger contacts or stylus contacts) or to multiplesimultaneous contacts (e.g., “multitouch”/multiple finger contacts). Insome embodiments, contact/motion module 130 and display controller 156detect contact on a touchpad.

Contact/motion module 130 optionally detects a gesture input by a user.Different gestures on the touch-sensitive surface have different contactpatterns (e.g., different motions, timings, and/or intensities ofdetected contacts). Thus, a gesture is, optionally, detected bydetecting a particular contact pattern. For example, detecting a fingertap gesture includes detecting a finger-down event followed by detectinga finger-up (lift off) event at the same position (or substantially thesame position) as the finger-down event (e.g., at the position of anicon). As another example, detecting a finger swipe gesture on thetouch-sensitive surface includes detecting a finger-down event followedby detecting one or more finger-dragging events, and subsequentlyfollowed by detecting a finger-up (lift off) event. Similarly, tap,swipe, drag, and other gestures are optionally detected for a stylus bydetecting a particular contact pattern for the stylus.

Graphics module 132 includes various known software components forrendering and displaying graphics on touch-sensitive display system 112or other display, including components for changing the visual impact(e.g., brightness, transparency, saturation, contrast or other visualproperty) of graphics that are displayed. As used herein, the term“graphics” includes any object that can be displayed to a user,including without limitation text, web pages, icons (such asuser-interface objects including soft keys), digital images, videos,animations and the like.

In some embodiments, graphics module 132 stores data representinggraphics to be used. Each graphic is, optionally, assigned acorresponding code. Graphics module 132 receives, from applicationsetc., one or more codes specifying graphics to be displayed along with,if necessary, coordinate data and other graphic property data, and thengenerates screen image data to output to display controller 156.

Haptic feedback module 133 includes various software components forgenerating instructions (e.g., used by haptic feedback controller 161)to produce tactile outputs using tactile output generator(s) 167 at oneor more locations on device 100 in response to user interactions withdevice 100.

Text input module 134, which is, optionally, a component of graphicsmodule 132, provides soft keyboards for entering text in variousapplications (e.g., contacts 137, e-mail 140, IM 141, browser 147, andany other application that needs text input).

GPS module 135 determines the location of the device and provides thisinformation for use in various applications (e.g., to telephone 138 foruse in location-based dialing, to camera 143 as picture/video metadata,and to applications that provide location-based services such as weatherwidgets, local yellow page widgets, and map/navigation widgets).

Applications 136 optionally include the following modules (or sets ofinstructions), or a subset or superset thereof:

-   -   contacts module 137 (sometimes called an address book or contact        list);    -   telephone module 138;    -   video conferencing module 139;    -   e-mail client module 140;    -   instant messaging (IM) module 141;    -   workout support module 142;    -   camera module 143 for still and/or video images;    -   image management module 144;    -   browser module 147;    -   calendar module 148;    -   widget modules 149, which optionally include one or more of:        weather widget 149-1, stocks widget 149-2, calculator widget        149-3, alarm clock widget 149-4, dictionary widget 149-5, and        other widgets obtained by the user, as well as user-created        widgets 149-6;    -   widget creator module 150 for making user-created widgets 149-6;    -   search module 151;    -   video and music player module 152, which is, optionally, made up        of a video player module and a music player module;    -   notes module 153;    -   map module 154; and/or    -   online video module 155.

Examples of other applications 136 that are, optionally, stored inmemory 102 include other word processing applications, other imageediting applications, drawing applications, presentation applications,JAVA-enabled applications, encryption, digital rights management, voicerecognition, and voice replication.

In conjunction with touch-sensitive display system 112, displaycontroller 156, contact module 130, graphics module 132, and text inputmodule 134, contacts module 137 includes executable instructions tomanage an address book or contact list (e.g., stored in applicationinternal state 192 of contacts module 137 in memory 102 or memory 370),including: adding name(s) to the address book; deleting name(s) from theaddress book; associating telephone number(s), e-mail address(es),physical address(es) or other information with a name; associating animage with a name; categorizing and sorting names; providing telephonenumbers and/or e-mail addresses to initiate and/or facilitatecommunications by telephone 138, video conference 139, e-mail 140, or IM141; and so forth.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch-sensitive display system 112, display controller156, contact module 130, graphics module 132, and text input module 134,telephone module 138 includes executable instructions to enter asequence of characters corresponding to a telephone number, access oneor more telephone numbers in address book 137, modify a telephone numberthat has been entered, dial a respective telephone number, conduct aconversation and disconnect or hang up when the conversation iscompleted. As noted above, the wireless communication optionally usesany of a plurality of communications standards, protocols andtechnologies.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch-sensitive display system 112, display controller156, optical sensor(s) 164, optical sensor controller 158, contactmodule 130, graphics module 132, text input module 134, contact list137, and telephone module 138, videoconferencing module 139 includesexecutable instructions to initiate, conduct, and terminate a videoconference between a user and one or more other participants inaccordance with user instructions.

In conjunction with RF circuitry 108, touch-sensitive display system112, display controller 156, contact module 130, graphics module 132,and text input module 134, e-mail client module 140 includes executableinstructions to create, send, receive, and manage e-mail in response touser instructions. In conjunction with image management module 144,e-mail client module 140 makes it very easy to create and send e-mailswith still or video images taken with camera module 143.

In conjunction with RF circuitry 108, touch-sensitive display system112, display controller 156, contact module 130, graphics module 132,and text input module 134, the instant messaging module 141 includesexecutable instructions to enter a sequence of characters correspondingto an instant message, to modify previously entered characters, totransmit a respective instant message (for example, using a ShortMessage Service (SMS) or Multimedia Message Service (MMS) protocol fortelephony-based instant messages or using XMPP, SIMPLE, Apple PushNotification Service (APNs) or IMPS for Internet-based instantmessages), to receive instant messages and to view received instantmessages. In some embodiments, transmitted and/or received instantmessages optionally include graphics, photos, audio files, video filesand/or other attachments as are supported in a MMS and/or an EnhancedMessaging Service (EMS). As used herein, “instant messaging” refers toboth telephony-based messages (e.g., messages sent using SMS or MMS) andInternet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs,or IMPS).

In conjunction with RF circuitry 108, touch-sensitive display system112, display controller 156, contact module 130, graphics module 132,text input module 134, GPS module 135, map module 154, and music playermodule 146, workout support module 142 includes executable instructionsto create workouts (e.g., with time, distance, and/or calorie burninggoals); communicate with workout sensors (in sports devices and smartwatches); receive workout sensor data; calibrate sensors used to monitora workout; select and play music for a workout; and display, store andtransmit workout data.

In conjunction with touch-sensitive display system 112, displaycontroller 156, optical sensor(s) 164, optical sensor controller 158,contact module 130, graphics module 132, and image management module144, camera module 143 includes executable instructions to capture stillimages or video (including a video stream) and store them into memory102, modify characteristics of a still image or video, and/or delete astill image or video from memory 102.

In conjunction with touch-sensitive display system 112, displaycontroller 156, contact module 130, graphics module 132, text inputmodule 134, and camera module 143, image management module 144 includesexecutable instructions to arrange, modify (e.g., edit), or otherwisemanipulate, label, delete, present (e.g., in a digital slide show oralbum), and store still and/or video images.

In conjunction with RF circuitry 108, touch-sensitive display system112, display system controller 156, contact module 130, graphics module132, and text input module 134, browser module 147 includes executableinstructions to browse the Internet in accordance with userinstructions, including searching, linking to, receiving, and displayingweb pages or portions thereof, as well as attachments and other fileslinked to web pages.

In conjunction with RF circuitry 108, touch-sensitive display system112, display system controller 156, contact module 130, graphics module132, text input module 134, e-mail client module 140, and browser module147, calendar module 148 includes executable instructions to create,display, modify, and store calendars and data associated with calendars(e.g., calendar entries, to do lists, etc.) in accordance with userinstructions.

In conjunction with RF circuitry 108, touch-sensitive display system112, display system controller 156, contact module 130, graphics module132, text input module 134, and browser module 147, widget modules 149are mini-applications that are, optionally, downloaded and used by auser (e.g., weather widget 149-1, stocks widget 149-2, calculator widget149-3, alarm clock widget 149-4, and dictionary widget 149-5) or createdby the user (e.g., user-created widget 149-6). In some embodiments, awidget includes an HTML (Hypertext Markup Language) file, a CSS(Cascading Style Sheets) file, and a JavaScript file. In someembodiments, a widget includes an XML (Extensible Markup Language) fileand a JavaScript file (e.g., Yahoo! Widgets).

In conjunction with RF circuitry 108, touch-sensitive display system112, display system controller 156, contact module 130, graphics module132, text input module 134, and browser module 147, the widget creatormodule 150 includes executable instructions to create widgets (e.g.,turning a user-specified portion of a web page into a widget).

In conjunction with touch-sensitive display system 112, display systemcontroller 156, contact module 130, graphics module 132, and text inputmodule 134, search module 151 includes executable instructions to searchfor text, music, sound, image, video, and/or other files in memory 102that match one or more search criteria (e.g., one or more user-specifiedsearch terms) in accordance with user instructions.

In conjunction with touch-sensitive display system 112, display systemcontroller 156, contact module 130, graphics module 132, audio circuitry110, speaker 111, RF circuitry 108, and browser module 147, video andmusic player module 152 includes executable instructions that allow theuser to download and play back recorded music and other sound filesstored in one or more file formats, such as MP3 or AAC files, andexecutable instructions to display, present or otherwise play backvideos (e.g., on touch-sensitive display system 112, or on an externaldisplay connected wirelessly or via external port 124). In someembodiments, device 100 optionally includes the functionality of an MP3player, such as an iPod (trademark of Apple Inc.).

In conjunction with touch-sensitive display system 112, displaycontroller 156, contact module 130, graphics module 132, and text inputmodule 134, notes module 153 includes executable instructions to createand manage notes, to do lists, and the like in accordance with userinstructions.

In conjunction with RF circuitry 108, touch-sensitive display system112, display system controller 156, contact module 130, graphics module132, text input module 134, GPS module 135, and browser module 147, mapmodule 154 includes executable instructions to receive, display, modify,and store maps and data associated with maps (e.g., driving directions;data on stores and other points of interest at or near a particularlocation; and other location-based data) in accordance with userinstructions.

In conjunction with touch-sensitive display system 112, display systemcontroller 156, contact module 130, graphics module 132, audio circuitry110, speaker 111, RF circuitry 108, text input module 134, e-mail clientmodule 140, and browser module 147, online video module 155 includesexecutable instructions that allow the user to access, browse, receive(e.g., by streaming and/or download), play back (e.g., on the touchscreen 112, or on an external display connected wirelessly or viaexternal port 124), send an e-mail with a link to a particular onlinevideo, and otherwise manage online videos in one or more file formats,such as H.264. In some embodiments, instant messaging module 141, ratherthan e-mail client module 140, is used to send a link to a particularonline video.

Each of the above identified modules and applications correspond to aset of executable instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (i.e., sets of instructions) need notbe implemented as separate software programs, procedures or modules, andthus various subsets of these modules are, optionally, combined orotherwise re-arranged in various embodiments. In some embodiments,memory 102 optionally stores a subset of the modules and data structuresidentified above. Furthermore, memory 102 optionally stores additionalmodules and data structures not described above.

In some embodiments, device 100 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen and/or a touchpad. By using a touch screen and/ora touchpad as the primary input control device for operation of device100, the number of physical input control devices (such as push buttons,dials, and the like) on device 100 is, optionally, reduced.

The predefined set of functions that are performed exclusively through atouch screen and/or a touchpad optionally include navigation betweenuser interfaces. In some embodiments, the touchpad, when touched by theuser, navigates device 100 to a main, home, or root menu from any userinterface that is displayed on device 100. In such embodiments, a “menubutton” is implemented using a touchpad. In some other embodiments, themenu button is a physical push button or other physical input controldevice instead of a touchpad.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments. In some embodiments,memory 102 (in FIG. 1A) or 370 (FIG. 3A or 3B) includes event sorter 170(e.g., in operating system 126) and a respective application 136-1(e.g., any of the aforementioned applications 136, 137-155, 380-390).

Event sorter 170 receives event information and determines theapplication 136-1 and application view 191 of application 136-1 to whichto deliver the event information. Event sorter 170 includes eventmonitor 171 and event dispatcher module 174. In some embodiments,application 136-1 includes application internal state 192, whichindicates the current application view(s) displayed on touch-sensitivedisplay system 112 when the application is active or executing. In someembodiments, device/global internal state 157 is used by event sorter170 to determine which application(s) is (are) currently active, andapplication internal state 192 is used by event sorter 170 to determineapplication views 191 to which to deliver event information.

In some embodiments, application internal state 192 includes additionalinformation, such as one or more of: resume information to be used whenapplication 136-1 resumes execution, user interface state informationthat indicates information being displayed or that is ready for displayby application 136-1, a state queue for enabling the user to go back toa prior state or view of application 136-1, and a redo/undo queue ofprevious actions taken by the user.

Event monitor 171 receives event information from peripherals interface118. Event information includes information about a sub-event (e.g., auser touch on touch-sensitive display system 112, as part of amulti-touch gesture). Peripherals interface 118 transmits information itreceives from I/O subsystem 106 or a sensor, such as proximity sensor166, accelerometer(s) 168, and/or microphone 113 (through audiocircuitry 110). Information that peripherals interface 118 receives fromI/O subsystem 106 includes information from touch-sensitive displaysystem 112 or a touch-sensitive surface.

In some embodiments, event monitor 171 sends requests to the peripheralsinterface 118 at predetermined intervals. In response, peripheralsinterface 118 transmits event information. In other embodiments,peripheral interface 118 transmits event information only when there isa significant event (e.g., receiving an input above a predeterminednoise threshold and/or for more than a predetermined duration).

In some embodiments, event sorter 170 also includes a hit viewdetermination module 172 and/or an active event recognizer determinationmodule 173.

Hit view determination module 172 provides software procedures fordetermining where a sub-event has taken place within one or more views,when touch-sensitive display system 112 displays more than one view.Views are made up of controls and other elements that a user can see onthe display.

Another aspect of the user interface associated with an application is aset of views, sometimes herein called application views or userinterface windows, in which information is displayed and touch-basedgestures occur. The application views (of a respective application) inwhich a touch is detected optionally correspond to programmatic levelswithin a programmatic or view hierarchy of the application. For example,the lowest level view in which a touch is detected is, optionally,called the hit view, and the set of events that are recognized as properinputs are, optionally, determined based, at least in part, on the hitview of the initial touch that begins a touch-based gesture.

Hit view determination module 172 receives information related tosub-events of a touch-based gesture. When an application has multipleviews organized in a hierarchy, hit view determination module 172identifies a hit view as the lowest view in the hierarchy which shouldhandle the sub-event. In most circumstances, the hit view is the lowestlevel view in which an initiating sub-event occurs (i.e., the firstsub-event in the sequence of sub-events that form an event or potentialevent). Once the hit view is identified by the hit view determinationmodule, the hit view typically receives all sub-events related to thesame touch or input source for which it was identified as the hit view.

Active event recognizer determination module 173 determines which viewor views within a view hierarchy should receive a particular sequence ofsub-events. In some embodiments, active event recognizer determinationmodule 173 determines that only the hit view should receive a particularsequence of sub-events. In other embodiments, active event recognizerdetermination module 173 determines that all views that include thephysical location of a sub-event are actively involved views, andtherefore determines that all actively involved views should receive aparticular sequence of sub-events. In other embodiments, even if touchsub-events were entirely confined to the area associated with oneparticular view, views higher in the hierarchy would still remain asactively involved views.

Event dispatcher module 174 dispatches the event information to an eventrecognizer (e.g., event recognizer 180). In embodiments including activeevent recognizer determination module 173, event dispatcher module 174delivers the event information to an event recognizer determined byactive event recognizer determination module 173. In some embodiments,event dispatcher module 174 stores in an event queue the eventinformation, which is retrieved by a respective event receiver module182.

In some embodiments, operating system 126 includes event sorter 170.Alternatively, application 136-1 includes event sorter 170. In yet otherembodiments, event sorter 170 is a stand-alone module, or a part ofanother module stored in memory 102, such as contact/motion module 130.

In some embodiments, application 136-1 includes a plurality of eventhandlers 190 and one or more application views 191, each of whichincludes instructions for handling touch events that occur within arespective view of the application's user interface. Each applicationview 191 of the application 136-1 includes one or more event recognizers180. Typically, a respective application view 191 includes a pluralityof event recognizers 180. In other embodiments, one or more of eventrecognizers 180 are part of a separate module, such as a user interfacekit (not shown) or a higher level object from which application 136-1inherits methods and other properties. In some embodiments, a respectiveevent handler 190 includes one or more of: data updater 176, objectupdater 177, GUI updater 178, and/or event data 179 received from eventsorter 170. Event handler 190 optionally utilizes or calls data updater176, object updater 177 or GUI updater 178 to update the applicationinternal state 192. Alternatively or additionally, one or more of theapplication views 191 includes one or more respective event handlers190. Also, in some embodiments, one or more of data updater 176, objectupdater 177, and GUI updater 178 are included in a respectiveapplication view 191.

A respective event recognizer 180 receives event information (e.g.,event data 179) from event sorter 170, and identifies an event from theevent information. Event recognizer 180 includes event receiver 182 andevent comparator 184. In some embodiments, event recognizer 180 alsoincludes at least a subset of: metadata 183, and event deliveryinstructions 188 (which optionally include sub-event deliveryinstructions).

Event receiver 182 receives event information from event sorter 170. Theevent information includes information about a sub-event, for example, atouch or a touch movement. Depending on the sub-event, the eventinformation also includes additional information, such as location ofthe sub-event. When the sub-event concerns motion of a touch, the eventinformation optionally also includes speed and direction of thesub-event. In some embodiments, events include rotation of the devicefrom one orientation to another (e.g., from a portrait orientation to alandscape orientation, or vice versa), and the event informationincludes corresponding information about the current orientation (alsocalled device attitude) of the device.

Event comparator 184 compares the event information to predefined eventor sub-event definitions and, based on the comparison, determines anevent or sub-event, or determines or updates the state of an event orsub-event. In some embodiments, event comparator 184 includes eventdefinitions 186. Event definitions 186 contain definitions of events(e.g., predefined sequences of sub-events), for example, event 1(187-1), event 2 (187-2), and others. In some embodiments, sub-events inan event 187 include, for example, touch begin, touch end, touchmovement, touch cancellation, and multiple touching. In one example, thedefinition for event 1 (187-1) is a double tap on a displayed object.The double tap, for example, comprises a first touch (touch begin) onthe displayed object for a predetermined phase, a first lift-off (touchend) for a predetermined phase, a second touch (touch begin) on thedisplayed object for a predetermined phase, and a second lift-off (touchend) for a predetermined phase. In another example, the definition forevent 2 (187-2) is a dragging on a displayed object. The dragging, forexample, comprises a touch (or contact) on the displayed object for apredetermined phase, a movement of the touch across touch-sensitivedisplay system 112, and lift-off of the touch (touch end). In someembodiments, the event also includes information for one or moreassociated event handlers 190.

In some embodiments, event definition 187 includes a definition of anevent for a respective user-interface object. In some embodiments, eventcomparator 184 performs a hit test to determine which user-interfaceobject is associated with a sub-event. For example, in an applicationview in which three user-interface objects are displayed ontouch-sensitive display system 112, when a touch is detected ontouch-sensitive display system 112, event comparator 184 performs a hittest to determine which of the three user-interface objects isassociated with the touch (sub-event). If each displayed object isassociated with a respective event handler 190, the event comparatoruses the result of the hit test to determine which event handler 190should be activated. For example, event comparator 184 selects an eventhandler associated with the sub-event and the object triggering the hittest.

In some embodiments, the definition for a respective event 187 alsoincludes delayed actions that delay delivery of the event informationuntil after it has been determined whether the sequence of sub-eventsdoes or does not correspond to the event recognizer's event type.

When a respective event recognizer 180 determines that the series ofsub-events do not match any of the events in event definitions 186, therespective event recognizer 180 enters an event impossible, eventfailed, or event ended state, after which it disregards subsequentsub-events of the touch-based gesture. In this situation, other eventrecognizers, if any, that remain active for the hit view continue totrack and process sub-events of an ongoing touch-based gesture.

In some embodiments, a respective event recognizer 180 includes metadata183 with configurable properties, flags, and/or lists that indicate howthe event delivery system should perform sub-event delivery to activelyinvolved event recognizers. In some embodiments, metadata 183 includesconfigurable properties, flags, and/or lists that indicate how eventrecognizers interact, or are enabled to interact, with one another. Insome embodiments, metadata 183 includes configurable properties, flags,and/or lists that indicate whether sub-events are delivered to varyinglevels in the view or programmatic hierarchy.

In some embodiments, a respective event recognizer 180 activates eventhandler 190 associated with an event when one or more particularsub-events of an event are recognized. In some embodiments, a respectiveevent recognizer 180 delivers event information associated with theevent to event handler 190. Activating an event handler 190 is distinctfrom sending (and deferred sending) sub-events to a respective hit view.In some embodiments, event recognizer 180 throws a flag associated withthe recognized event, and event handler 190 associated with the flagcatches the flag and performs a predefined process.

In some embodiments, event delivery instructions 188 include sub-eventdelivery instructions that deliver event information about a sub-eventwithout activating an event handler. Instead, the sub-event deliveryinstructions deliver event information to event handlers associated withthe series of sub-events or to actively involved views. Event handlersassociated with the series of sub-events or with actively involved viewsreceive the event information and perform a predetermined process.

In some embodiments, data updater 176 creates and updates data used inapplication 136-1. For example, data updater 176 updates the telephonenumber used in contacts module 137, or stores a video file used in videoplayer module 145. In some embodiments, object updater 177 creates andupdates objects used in application 136-1. For example, object updater176 creates a new user-interface object or updates the position of auser-interface object. GUI updater 178 updates the GUI. For example, GUIupdater 178 prepares display information and sends it to graphics module132 for display on a touch-sensitive display.

In some embodiments, event handler(s) 190 includes or has access to dataupdater 176, object updater 177, and GUI updater 178. In someembodiments, data updater 176, object updater 177, and GUI updater 178are included in a single module of a respective application 136-1 orapplication view 191. In other embodiments, they are included in two ormore software modules.

It shall be understood that the foregoing discussion regarding eventhandling of user touches on touch-sensitive displays also applies toother forms of user inputs to operate multifunction devices 100 withinput-devices, not all of which are initiated on touch screens. Forexample, mouse movement and mouse button presses, optionally coordinatedwith single or multiple keyboard presses or holds; contact movementssuch as taps, drags, scrolls, etc., on touch-pads; pen stylus inputs;movement of the device; oral instructions; detected eye movements;biometric inputs; and/or any combination thereof are optionally utilizedas inputs corresponding to sub-events which define an event to berecognized.

FIG. 2 illustrates a portable multifunction device 100 having a touchscreen (e.g., touch-sensitive display system 112, FIG. 1A) in accordancewith some embodiments. The touch screen optionally displays one or moregraphics within user interface (UI) 200. In these embodiments, as wellas others described below, a user is enabled to select one or more ofthe graphics by making a gesture on the graphics, for example, with oneor more fingers 202 (not drawn to scale in the figure) or one or morestyluses 203 (not drawn to scale in the figure). In some embodiments,selection of one or more graphics occurs when the user breaks contactwith the one or more graphics. In some embodiments, the gestureoptionally includes one or more taps, one or more swipes (from left toright, right to left, upward and/or downward) and/or a rolling of afinger (from right to left, left to right, upward and/or downward) thathas made contact with device 100. In some implementations orcircumstances, inadvertent contact with a graphic does not select thegraphic. For example, a swipe gesture that sweeps over an applicationicon optionally does not select the corresponding application when thegesture corresponding to selection is a tap.

Device 100 optionally also includes one or more physical buttons, suchas “home” or menu button 204. As described previously, menu button 204is, optionally, used to navigate to any application 136 in a set ofapplications that are, optionally executed on device 100. Alternatively,in some embodiments, the menu button is implemented as a soft key in aGUI displayed on the touch-screen display.

In some embodiments, device 100 includes the touch-screen display, menubutton 204, push button 206 for powering the device on/off and lockingthe device, volume adjustment button(s) 208, Subscriber Identity Module(SIM) card slot 210, head set jack 212, and docking/charging externalport 124. Push button 206 is, optionally, used to turn the power on/offon the device by depressing the button and holding the button in thedepressed state for a predefined time interval; to lock the device bydepressing the button and releasing the button before the predefinedtime interval has elapsed; and/or to unlock the device or initiate anunlock process. In some embodiments, device 100 also accepts verbalinput for activation or deactivation of some functions throughmicrophone 113. Device 100 also, optionally, includes one or morecontact intensity sensors 165 for detecting intensity of contacts ontouch-sensitive display system 112 and/or one or more tactile outputgenerators 167 for generating tactile outputs for a user of device 100.

FIG. 3A is a block diagram of an exemplary multifunction device 300 witha display 340 and a touch-sensitive surface 355 in accordance with someembodiments. Device 300 need not be portable. In some embodiments,device 300 is a laptop computer, a desktop computer, a tablet computer,a multimedia player device, a navigation device, an educational device(such as a child's learning toy), a gaming system, or a control device(e.g., a home or industrial controller).

Device 300 typically includes one or more processing units (also calledherein CPU's and processors) 310, one or more network or othercommunications interfaces 360, memory 370, and one or more communicationbuses 320 for interconnecting these components. Communication buses 320optionally include circuitry (sometimes called a chipset) thatinterconnects and controls communications between system components.

Device 300 includes input/output (I/O) interface 330 comprising display340, which is a touch-screen display in some embodiments. In someembodiments, I/O interface 330 is coupled to the one or more processingunits 310 and/or memory 370 through a peripherals interface (not shownin FIG. 3A). I/O interface 330 also optionally includes a keyboardand/or mouse (or other pointing device) 350 and touchpad 355, tactileoutput generator 357 for generating tactile outputs on device 300 (e.g.,similar to tactile output generator(s) 167 described above withreference to FIG. 1A), contact intensity sensors 359, similar to contactintensity sensor(s) 165 described above with reference to FIG. 1A,and/or other sensors 361 (e.g., optical, acceleration, proximity, and/ortouch-sensitive sensors).

Memory 370 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM or other random access solid state memory devices; andoptionally includes non-volatile memory, such as one or more magneticdisk storage devices, optical disk storage devices, flash memorydevices, or other non-volatile solid state storage devices. Memory 370optionally includes one or more storage devices remotely located fromCPU(s) 310.

In some embodiments, memory 370 stores programs, modules, and datastructures analogous to the programs, modules, and data structuresstored in memory 102 of portable multifunction device 100 (FIG. 1A), ora subset thereof. Furthermore, memory 370 optionally stores additionalprograms, modules, and data structures not present in memory 102 ofportable multifunction device 100. For example, memory 370 of device 300optionally stores drawing module 380, presentation module 382, wordprocessing module 384, website creation module 386, disk authoringmodule 388, and/or spreadsheet module 390, while memory 102 of portablemultifunction device 100 (FIG. 1A) optionally does not store thesemodules.

In some embodiments, the operating system 126 includes one or moredevice modules 127 (also called device drivers). The one or more devicemodules 127 include software components that operate or controlparticular hardware devices included in or in communication with themultifunction device 300 (e.g., touchpad 355, tactile output generators357, intensity sensors 359, and other sensors 361 shown in FIG. 3Aand/or trackpad 332 and its components shown in FIG. 3B).

Each of the above identified elements in FIG. 3A are, optionally, storedin one or more of the previously mentioned memory devices. Each of theabove identified modules corresponds to a set of instructions forperforming a function described above. The above identified modules orprograms (i.e., sets of instructions) need not be implemented asseparate software programs, procedures or modules, and thus varioussubsets of these modules are, optionally, combined or otherwisere-arranged in various embodiments. In some embodiments, memory 370optionally stores a subset of the modules and data structures identifiedabove. Furthermore, memory 370 optionally stores additional modules anddata structures not described above.

FIG. 3B is a block diagram of an exemplary multifunction device 300 witha display and a touch-sensitive surface in accordance with someembodiments. FIG. 3B is similar to FIG. 3A except that the multifunctiondevice 300 in FIG. 3B includes trackpad 332 that is coupled to thePeripherals Interface 118 through External Port 124 (e.g., using a USBport or any other wired communication protocols). Alternatively, thetrackpad 332 may be coupled to the Peripherals Interface 118 through RFcircuitry 108 (e.g., using Bluetooth or any other form of wirelesscommunication protocols). In some embodiments, the trackpad 332 islocated outside an enclosure for the device 300. In comparison, thetouchpad 355 is typically integrated with the enclosure for the device300. In some embodiments, the trackpad 332 is integrated with theenclosure for the device 300. In some embodiments, the device 300includes the trackpad 332 in addition to the touchpad 355 (and/ortactile output generator(s) 357). In some embodiments, the device 300includes the trackpad 332 instead of the touchpad 355 (and/or tactileoutput generator(s) 357).

The trackpad 332 includes touch-sensitive surface 334. In someembodiments, the trackpad 332 includes one or more intensity sensors 336to detect intensity of contacts on the touch-sensitive surface 334. Insome embodiments, the trackpad 332 includes one or more tactile outputgenerators 338. In some embodiments, the trackpad 332 includes one ormore processors 342 to process signals from the touch-sensitive surface334 and the one or more intensity sensors 336, if included, and controloperations of the one or more tactile output generators 338, ifincluded. In some embodiments, the one or more processors 342 includeIntensity Sensors Controller 159 and/or Haptic Feedback Controller 161described above with respect to FIG. 1A. In some embodiments, the one ormore processors 342 are configured to detect contact (and any movementor breaking of the contact) on the touch-sensitive surface 334 andconvert the detected contact into predefined user inputs, such aspredefined touch gestures, which are processed by software applicationsto initiate interaction with user-interface objects (e.g., one or moresoft keys, icons, web pages or images) that are displayed ontouch-sensitive display system 112. In some embodiments, the touchpad355 also includes a touch-sensitive surface and one or more processorssimilar to the one or more processors 342 of the trackpad 332.

The trackpad 332 includes Communication Module 344 to transmitinformation to the one or more processing units 310 through thePeripherals Interface 118 and/or receive instructions from the one ormore processing units 310 and/or the Peripherals Interface 118.

In some embodiments, the trackpad 332 includes other sensors 346, suchas a proximity sensor, an accelerometer, etc.

FIG. 3C is a perspective view of an exemplary multifunction device 300with a touch-sensitive surface in accordance with some embodiments. Asshown in FIG. 3C, the device 300 may be a laptop computer. The device300 includes enclosure 302, display 340, touchpad 355, mouse 350, one ormore external ports 124, and one or more input or control devices 116(e.g., a power on/off button). The enclosure 302 forms at least aportion of an exterior of the device 300. In some embodiments, theenclosure 302 at least partially surrounds certain components, such asCPUs 310 and memory 370 of the device 300.

In some embodiments, the multifunction device shown in FIG. 3C includesone or more tactile output generators 357 as shown in FIG. 3A.

FIG. 3D is a perspective view of an input device 332 (e.g., a trackpad)incorporating a touch-sensitive surface separate from a multifunctiondevice in accordance with some embodiments. In some embodiments, theinput device 332 includes touch-sensitive surface 334 and one or moreof: one or more intensity sensors 336, one or more tactile outputgenerators 338, one or more processors 342, a communication module 344,and sensors 346, as shown in FIG. 3B. In some embodiments, the inputdevice 332 is coupled to the device 300 shown in FIG. 3C using wirelessor wired communication protocols (e.g., through the external port 124 ofthe device 300).

FIG. 3E is a simplified block diagram of an exemplary multifunctiondevice that includes a trackpad (e.g., trackpad 332 in FIG. 3B) inaccordance with some embodiments.

The trackpad 332 includes one or more of: a touch-sensitive surface 334,one or more intensity sensors 336, one or more tactile output generators338, and a communication module 344, as described above with respect toFIG. 3E. In some of the embodiments in which the trackpad 332 includesone or more processors 342, the trackpad 332 includes a firmware 348that includes instructions for execution by the one or more processors342 to process signals from the touch-sensitive surface 334 and/or theintensity sensors 336 and operate the tactile output generators 338. Insome embodiments, the one or more processors 342 process raw signalsfrom the touch-sensitive surface 334 and/or the intensity sensors 336and transmit processed information (e.g., coordinates of contacts andintensity of contacts, etc.) to the processors 310 for furtherprocessing (e.g., in accordance with the operating system 126, devicedrivers 127, and/or the applications 136), such as identifying a touchgesture and normalizing the intensity of contacts. In some embodiments,the processed information is sent via the peripherals interface 118 fromthe communication module 344 to the processors 310. In some embodiments,the processed information is transmitted by the processors 342 to theprocessors 310 without sending the raw signals from the touch-sensitivesurface 334 and/or the intensity sensors 336. In some embodiments, theone or more processors 342 further process the processed information toobtain mapped information (e.g., information for an identified gesture,normalized intensity, etc.) and sends the mapped information to theprocessors 310. This reduces the work load of the processors 310. Insome embodiments, the processors 310 are multi-purpose processing units(e.g., CPUs, APUs, etc.) and the processors 342 are application-specificprocessing units (e.g., application-specific integrated circuits(ASICs)) and/or field-programmable gate arrays (FPGA), which may be morepower efficient than multi-purpose processing units.

In some embodiments, information received through the peripheralsinterface 118 is processed by the processors 310 using the communicationmodule 128, and then using the device drivers 127. For example,information based on signals measured by the touch-sensitive surface 334is routed to a device driver that corresponds to the touch-sensitivesurface 334, and information based on signals measured by the one ormore intensity sensors 336 is routed to a device driver that correspondsto the one or more intensity sensors 336. In some embodiments, a singledevice driver that corresponds to the trackpad 332 includes instructionsfor handling the information based on signals measured by thetouch-sensitive surface 334 and the information based on signalsmeasured by the one or more intensity sensors 336.

In some embodiments, the processors 310 sends instructions to thetrackpad 332 (e.g., in accordance with the applications 136) to generatea tactile output using a device driver that corresponds to the tactileoutput generators 338.

In some embodiments, the device drivers 127 are separate from theoperating system 126 as illustrated in FIG. 3E. In some embodiments, thedevice drivers 127 are included in the operating system 126. In someembodiments, the communication module 128 is separate from the operatingsystem 126. In some embodiments, the communication module 128 isincluded in the operating system 126.

FIG. 3F is a simplified block diagram illustrating architecture of anexemplary multifunction device in accordance with some embodiments.

Hardware (e.g., electronic circuitry) 352 of the device is at the baselevel of the architecture. Hardware 352 can include various hardwareinterface components, such as the components depicted in FIGS. 1A, 3Aand/or 3B. For example, Hardware 352 includes Touch-Sensitive Surface334 and the one or more Intensity Sensors 336 described above withrespect to FIGS. 1A, 3A, 3B, and 3E. At least some of the other elements(348, 126, 356, and 136) of the architecture are software procedures, orportions of software procedures, that process inputs received fromHardware 352 and generate various outputs that are presented through ahardware user interface (e.g., one or more of a display, speakers,device vibration actuator, etc.).

Firmware 348 is used to communicate with Hardware 352. In someembodiments, Firmware 348 includes device drivers. Firmware 348 is usedto receive and process input data received from Hardware 352. In someembodiments, at least a portion of Hardware 352 and Firmware 348 isimplemented in Trackpad 332.

In some embodiments, Operating System (“OS”) 126 communicates withFirmware 348. OS 126 can process raw input data or processed datareceived from Firmware 348.

Application Programming Interfaces (“APIs”) 356 are software proceduresthat are used to communicate with OS 126 (or Device Drivers 127 of FIG.3E). In some embodiments, APIs 356 are included in the device'soperating system, but at a level above its core OS. APIs 356 aredesigned for use by Applications 136 running on the electronic devicesor apparatuses discussed herein. Application software 136 includes oneor more applications 136 (FIGS. 1A, 3A, and 3B).

While each layer in the architecture can utilize the layer underneathit, that is not always required. For example, in some embodiments,Applications 136 may directly communicate with OS 126. In someembodiments, Applications 136 and API 356 cannot directly accessFirmware 348 or Hardware 352, as these layers are considered private.Applications 136 usually direct calls API 356, which in turn, accessesOS 126, Firmware 348, and Hardware 352.

FIG. 3G is a block diagram illustrating data structures used by anexemplary multifunction device in accordance with some embodiments.

In FIG. 3G, application 136 communicates with contact motion module 130.In some embodiments, contact motion module 130 is implemented in OS 126.In some embodiments, contact motion module 130 is separate from OS 126.In some embodiments, contact motion module 130 is implemented infirmware 348 (FIG. 3E).

In some embodiments, contact module 130 includes event sorter 170 (FIG.1B). In some embodiments, contact intensity module 130 includes contactintensity module 175 that detects intensity of a contact on atouch-sensitive surface.

In some embodiments, contact motion module 130 stores a plurality ofintensity models 454. In some embodiments, the plurality of intensitymodels 454 (e.g., two or more intensity models, or alternatively threeor more intensity models) is used by contact intensity module 175. Arespective intensity model in the plurality of intensity models 454includes one or more of the following, or a superset or subset thereof:

-   -   Intensity Model Identifier 456, which identifies (typically        uniquely identifies) an intensity model (see Appendix A for        exemplary intensity models and corresponding intensity model        identifiers);    -   Use Info 458-1, which includes information that identifies        conditions for which the corresponding intensity model is        configured for use and/or conditions for which the corresponding        intensity model is prohibited for use (e.g., for particular        applications identified by Application Identity 487, and/or for        particular Regions 488 of a user interface and/or a        touch-sensitive surface), and/or one or more priorities 489        associated with the corresponding intensity model; and    -   Intensity Thresholds 464, which is described in further detail        below.

In some embodiments, intensity thresholds 464 include one or moreintensity thresholds for one or more intensity stages (e.g., 466-1,466-2, etc.). For example, intensity thresholds 464 include anactivation threshold 468-1 for entering a respective intensity stage. Insome embodiments, intensity thresholds 464 include a release threshold468-2 for exiting from the respective intensity stage. In someembodiments, the release threshold 468-2 for the respective intensitystage is identical to the activation threshold 468-1 for the respectiveintensity stage. In some embodiments, the release threshold 468-2 forthe respective intensity stage is distinct from the activation threshold468-1 for the respective intensity stage. In some embodiments, intensitythresholds 464 include one or more transition intensity thresholds468-3. For example, intensity thresholds 464 may include a transitionintensity threshold associated with the activation threshold 468-1,which is used to indicate that a contact of certain intensity is in atransition range from the activation threshold 468-1. Additionally oralternatively, intensity thresholds 464 may include a transitionintensity threshold associated with the release threshold 468-2, whichis used to indicate that a contact of certain intensity is in atransition range from the release threshold 468-2. In some embodiments,intensity thresholds 464 include tactile output parameters 468-2, whichindicate whether or not a tactile output is to be generated thatintensity of a contact crosses one of the intensity thresholds and, if atactile output is to be generated, what type of a tactile output is tobe generated (e.g., intensity, duration, and waveform of the tactileoutput).

In some embodiments, information sent by application 136 to contactmotion module 130 (especially contact intensity module 175 of contactmotion module 130) includes one or more of the following, or a subset orsuperset thereof:

-   -   Intensity Model Identifier 490, which identifies which intensity        model is to be used by contact intensity module 175;    -   Tracking Region 492, which identifies a region for which a        particular intensity model is to be used;    -   Tactile Output Trigger 494, which initiates generation of a        tactile output by one or more tactile output generators        controlled by contact motion module 130;    -   Application Identity 496, which identifies application 136 that        sends the information; and    -   Device Identifier 498, which identifies an input device (e.g., a        touchpad or a trackpad) that will use the identified intensity        model and/or generate a tactile output.

In some embodiments, information sent by contact motion module 130(especially contact intensity module 175 of contact motion module 130)includes one or more of the following, or a subset or superset thereof:

-   -   Availability Information 474, which indicates whether an input        device (e.g., a touch pad or a trackpad) is configured to        provide intensity information;    -   Characterization Parameters 476, such as Intensity Stage 478        and/or one or more Progress Values 480 (e.g., Stage Progress        Value 482 and Transition Progress Value 484);    -   Intensity Model Identifier 485, which indicates an intensity        model used by the contact intensity module 175 (e.g., for        determining characterization parameters 476);    -   Device Identifier 486, which identifies an input device from        which intensity signals have been received (e.g., for        determining characterization parameters 476); and    -   Other Touch Information (e.g., gesture type, number of contacts,        time stamp, etc.).

These information to and/or from contact motion module 130 need not betransmitted concurrently. For example, in some embodiments, availabilityinformation 474 is sent, separately from characterization parameters476, by contact intensity module 175. In another example, tactile outputtrigger 494 is transmitted separately from tracking region 492, in someembodiments.

In some embodiments, information exchanged between application 136 andcontact/motion module 130 is transmitted through Application ProgrammingInterface(s) 356 (FIG. 3F).

Although FIG. 3G illustrates communication between contact motion module130 and application 136, contact motion module 130 can be incommunication with multiple different applications 136 (e.g., contactsmodule 137, telephone module 138, video conference module 139, e-mailclient module 140, instant messaging module 141, workout support module142, camera module 143, image management module 144, browser module 147,calendar module 148, widget modules 149, search module 151, video andmusic player module 152, drawing module 380, presentation module 382,word processing module 384, website creation module 386, disk authoringmodule 388, spreadsheet module 390) that are configured to receivecontact information (e.g., touch events that include contactinformation). In some embodiments, contact motion module 130 is also incommunication with third party applications. In some embodiments,contact motion module 130 controls, for multiple applications, how pressinputs are interpreted (e.g., changing intensity thresholds). Thisallows press inputs to be interpreted consistently across the multipleapplications (e.g., providing consistent trackpad configurations, and/orproviding consistent tactile outputs).

Attention is now directed towards embodiments of user interfaces (“UI”)that are, optionally, implemented on portable multifunction device 100.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on portable multifunction device 100 in accordance withsome embodiments. Similar user interfaces are, optionally, implementedon device 300. In some embodiments, user interface 400 includes thefollowing elements, or a subset or superset thereof:

-   -   Signal strength indicator(s) 402 for wireless communication(s),        such as cellular and Wi-Fi signals;    -   Time 404;    -   Bluetooth indicator 405;    -   Battery status indicator 406;    -   Tray 408 with icons for frequently used applications, such as:        -   Icon 416 for telephone module 138, labeled “Phone,” which            optionally includes an indicator 414 of the number of missed            calls or voicemail messages;        -   Icon 418 for e-mail client module 140, labeled “Mail,” which            optionally includes an indicator 410 of the number of unread            e-mails;        -   Icon 420 for browser module 147, labeled “Browser;” and        -   Icon 422 for video and music player module 152, also            referred to as iPod (trademark of Apple Inc.) module 152,            labeled “iPod;” and    -   Icons for other applications, such as:        -   Icon 424 for IM module 141, labeled “Messages;”        -   Icon 426 for calendar module 148, labeled “Calendar;”        -   Icon 428 for image management module 144, labeled “Photos;”        -   Icon 430 for camera module 143, labeled “Camera;”        -   Icon 432 for online video module 155, labeled “Online            Video;”        -   Icon 434 for stocks widget 149-2, labeled “Stocks;”        -   Icon 436 for map module 154, labeled “Maps;”        -   Icon 438 for weather widget 149-1, labeled “Weather;”        -   Icon 440 for alarm clock widget 149-4, labeled “Clock;”        -   Icon 442 for workout support module 142, labeled “Workout            Support;”        -   Icon 444 for notes module 153, labeled “Notes;” and        -   Icon 446 for a settings application or module, which            provides access to settings for device 100 and its various            applications 136.

It should be noted that the icon labels illustrated in FIG. 4A aremerely exemplary. For example, in some embodiments, icon 422 for videoand music player module 152 is labeled “Music” or “Music Player.” Otherlabels are, optionally, used for various application icons. In someembodiments, a label for a respective application icon includes a nameof an application corresponding to the respective application icon. Insome embodiments, a label for a particular application icon is distinctfrom a name of an application corresponding to the particularapplication icon.

FIG. 4B illustrates an exemplary user interface on a device (e.g.,device 300, FIG. 3A or 3B) with a touch-sensitive surface 451 (e.g., atablet or touchpad 355, FIG. 3A or 3B) that is separate from the display450. Device 300 also, optionally, includes one or more contact intensitysensors (e.g., one or more of sensors 359) for detecting intensity ofcontacts on touch-sensitive surface 451 and/or one or more tactileoutput generators 357 for generating tactile outputs for a user ofdevice 300.

FIG. 4B illustrates an exemplary user interface on a device (e.g.,device 300, FIG. 3A or 3B) with a touch-sensitive surface 451 (e.g., atablet or touchpad 355, FIG. 3A or 3B) that is separate from the display450. Many of the examples that follow will be given with reference to adevice that detects inputs on a touch-sensitive surface that is separatefrom the display, as shown in FIG. 4B. In some embodiments, thetouch-sensitive surface (e.g., 451 in FIG. 4B) has a primary axis (e.g.,452 in FIG. 4B) that corresponds to a primary axis (e.g., 453 in FIG.4B) on the display (e.g., 450). In accordance with these embodiments,the device detects contacts (e.g., 460 and 462 in FIG. 4B) with thetouch-sensitive surface 451 at locations that correspond to respectivelocations on the display (e.g., in FIG. 4B, 460 corresponds to 468 and462 corresponds to 470). In this way, user inputs (e.g., contacts 460and 462, and movements thereof) detected by the device on thetouch-sensitive surface (e.g., 451 in FIG. 4B) are used by the device tomanipulate the user interface on the display (e.g., 450 in FIG. 4B) ofthe multifunction device when the touch-sensitive surface is separatefrom the display. It should be understood that similar methods are,optionally, used for other user interfaces described herein.

Additionally, while the following examples are given primarily withreference to finger inputs (e.g., finger contacts, finger tap gestures,finger swipe gestures, etc.), it should be understood that, in someembodiments, one or more of the finger inputs are replaced with inputfrom another input device (e.g., a mouse based input or a stylus input).For example, a swipe gesture is, optionally, replaced with a mouse click(e.g., instead of a contact) followed by movement of the cursor alongthe path of the swipe (e.g., instead of movement of the contact). Asanother example, a tap gesture is, optionally, replaced with a mouseclick while the cursor is located over the location of the tap gesture(e.g., instead of detection of the contact followed by ceasing to detectthe contact). Similarly, when multiple user inputs are simultaneouslydetected, it should be understood that multiple computer mice are,optionally, used simultaneously, or a mouse and finger contacts are,optionally, used simultaneously.

As used herein, the term “focus selector” refers to an input elementthat indicates a current part of a user interface with which a user isinteracting. In some implementations that include a cursor or otherlocation marker, the cursor acts as a “focus selector,” so that when aninput (e.g., a press input) is detected on a touch-sensitive surface(e.g., touchpad 355 in FIG. 3A or 3B or touch-sensitive surface 451 inFIG. 4B) while the cursor is over a particular user interface element(e.g., a button, window, slider or other user interface element), theparticular user interface element is adjusted in accordance with thedetected input. In some implementations that include a touch-screendisplay (e.g., touch-sensitive display system 112 in FIG. 1A or thetouch screen in FIG. 4A) that enables direct interaction with userinterface elements on the touch-screen display, a detected contact onthe touch-screen acts as a “focus selector,” so that when an input(e.g., a press input by the contact) is detected on the touch-screendisplay at a location of a particular user interface element (e.g., abutton, window, slider or other user interface element), the particularuser interface element is adjusted in accordance with the detectedinput. In some implementations, focus is moved from one region of a userinterface to another region of the user interface without correspondingmovement of a cursor or movement of a contact on a touch-screen display(e.g., by using a tab key or arrow keys to move focus from one button toanother button); in these implementations, the focus selector moves inaccordance with movement of focus between different regions of the userinterface. Without regard to the specific form taken by the focusselector, the focus selector is generally the user interface element (orcontact on a touch-screen display) that is controlled by the user so asto communicate the user's intended interaction with the user interface(e.g., by indicating, to the device, the element of the user interfacewith which the user is intending to interact). For example, the locationof a focus selector (e.g., a cursor, a contact, or a selection box) overa respective button while a press input is detected on thetouch-sensitive surface (e.g., a touchpad or touch screen) will indicatethat the user is intending to activate the respective button (as opposedto other user interface elements shown on a display of the device).

As used in the specification and claims, the term “intensity” of acontact on a touch-sensitive surface refers to the force or pressure(force per unit area) of a contact (e.g., a finger contact or a styluscontact) on the touch-sensitive surface, or to a substitute (proxy) forthe force or pressure of a contact on the touch-sensitive surface. Theintensity of a contact has a range of values that includes at least fourdistinct values and more typically includes hundreds of distinct values(e.g., at least 256). Intensity of a contact is, optionally, determined(or measured) using various approaches and various sensors orcombinations of sensors. For example, one or more force sensorsunderneath or adjacent to the touch-sensitive surface are, optionally,used to measure force at various points on the touch-sensitive surface.In some implementations, force measurements from multiple force sensorsare combined (e.g., a weighted average or a sum) to determine anestimated force of a contact. Similarly, a pressure-sensitive tip of astylus is, optionally, used to determine a pressure of the stylus on thetouch-sensitive surface. Alternatively, the size of the contact areadetected on the touch-sensitive surface and/or changes thereto, thecapacitance of the touch-sensitive surface proximate to the contactand/or changes thereto, and/or the resistance of the touch-sensitivesurface proximate to the contact and/or changes thereto are, optionally,used as a substitute for the force or pressure of the contact on thetouch-sensitive surface. In some implementations, the substitutemeasurements for contact force or pressure are used directly todetermine whether an intensity threshold has been exceeded (e.g., theintensity threshold is described in units corresponding to thesubstitute measurements). In some implementations, the substitutemeasurements for contact force or pressure are converted to an estimatedforce or pressure and the estimated force or pressure is used todetermine whether an intensity threshold has been exceeded (e.g., theintensity threshold is a pressure threshold measured in units ofpressure). Using the intensity of a contact as an attribute of a userinput allows user access to additional device functionality that mayotherwise not be readily accessible by the user on a reduced-size devicewith limited real estate for displaying affordances (e.g., on atouch-sensitive display) and/or receiving user input (e.g., via atouch-sensitive display, a touch-sensitive surface, or aphysical/mechanical control such as a knob or a button).

In some embodiments, contact/motion module 130 uses a set of one or moreintensity thresholds to determine whether an operation has beenperformed by a user (e.g., to determine whether a user has “clicked” onan icon). In some embodiments, at least a subset of the intensitythresholds are determined in accordance with software parameters (e.g.,the intensity thresholds are not determined by the activation thresholdsof particular physical actuators and can be adjusted without changingthe physical hardware of device 100). For example, a mouse “click”threshold of a trackpad or touch-screen display can be set to any of alarge range of predefined thresholds values without changing thetrackpad or touch-screen display hardware. Additionally, in someimplementations a user of the device is provided with software settingsfor adjusting one or more of the set of intensity thresholds (e.g., byadjusting individual intensity thresholds and/or by adjusting aplurality of intensity thresholds at once with a system-level click“intensity” parameter).

As used in the specification and claims, the term “characteristicintensity” of a contact refers to a characteristic of the contact basedon one or more intensities of the contact. In some embodiments, thecharacteristic intensity is based on multiple intensity samples. Thecharacteristic intensity is, optionally, based on a predefined number ofintensity samples, or a set of intensity samples collected during apredetermined time period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10seconds) relative to a predefined event (e.g., after detecting thecontact, prior to detecting liftoff of the contact, before or afterdetecting a start of movement of the contact, prior to detecting an endof the contact, before or after detecting an increase in intensity ofthe contact, and/or before or after detecting a decrease in intensity ofthe contact). A characteristic intensity of a contact is, optionallybased on one or more of: a maximum value of the intensities of thecontact, a mean value of the intensities of the contact, an averagevalue of the intensities of the contact, a top 10 percentile value ofthe intensities of the contact, a value at the half maximum of theintensities of the contact, a value at the 90 percent maximum of theintensities of the contact, or the like. In some embodiments, theduration of the contact is used in determining the characteristicintensity (e.g., when the characteristic intensity is an average of theintensity of the contact over time). In some embodiments, thecharacteristic intensity is compared to a set of one or more intensitythresholds to determine whether an operation has been performed by auser. For example, the set of one or more intensity thresholds mayinclude a first intensity threshold and a second intensity threshold. Inthis example, a contact with a characteristic intensity that does notexceed the first threshold results in a first operation, a contact witha characteristic intensity that exceeds the first intensity thresholdand does not exceed the second intensity threshold results in a secondoperation, and a contact with a characteristic intensity that exceedsthe second intensity threshold results in a third operation. In someembodiments, a comparison between the characteristic intensity and oneor more intensity thresholds is used to determine whether or not toperform one or more operations (e.g., whether to perform a respectiveoption or forgo performing the respective operation) rather than beingused to determine whether to perform a first operation or a secondoperation.

In some embodiments, a portion of a gesture is identified for purposesof determining a characteristic intensity. For example, atouch-sensitive surface may receive a continuous swipe contacttransitioning from a start location and reaching an end location (e.g.,a drag gesture), at which point the intensity of the contact increases.In this example, the characteristic intensity of the contact at the endlocation may be based on only a portion of the continuous swipe contact,and not the entire swipe contact (e.g., only the portion of the swipecontact at the end location). In some embodiments, a smoothing algorithmmay be applied to the intensities of the swipe contact prior todetermining the characteristic intensity of the contact. For example,the smoothing algorithm optionally includes one or more of: anunweighted sliding-average smoothing algorithm, a triangular smoothingalgorithm, a median filter smoothing algorithm, and/or an exponentialsmoothing algorithm. In some circumstances, these smoothing algorithmseliminate narrow spikes or dips in the intensities of the swipe contactfor purposes of determining a characteristic intensity.

The user interface figures (e.g., FIGS. 5A-5TT) described belowoptionally include various intensity diagrams that show the currentintensity of the contact on the touch-sensitive surface relative to oneor more intensity thresholds (e.g., a contact detection intensitythreshold IT₀, a latching intensity threshold IT_(L), an activationintensity threshold IT_(A), and/or one or more other intensitythresholds). This intensity diagram is typically not part of thedisplayed user interface, but is provided to aid in the interpretationof the figures. In some embodiments, a light press intensity threshold(e.g., a lower intensity threshold) corresponds to an intensity at whichthe device will perform operations typically associated with clicking abutton of a physical mouse or a trackpad. In some embodiments, a deeppress intensity threshold (e.g., a higher intensity threshold)corresponds to an intensity at which the device will perform operationsthat are different from operations typically associated with clicking abutton of a physical mouse or a trackpad. In some embodiments, when acontact is detected with a characteristic intensity below the lightpress intensity threshold (e.g., and above a nominal contact-detectionintensity threshold IT₀ below which the contact is no longer detected),the device will move a focus selector in accordance with movement of thecontact on the touch-sensitive surface without performing an operationassociated with the light press intensity threshold or the deep pressintensity threshold. Generally, unless otherwise stated, these intensitythresholds are consistent between different sets of user interfacefigures.

In some embodiments, the response of the device to inputs detected bythe device depends on criteria based on the contact intensity during theinput. For example, for some “light press” inputs, the intensity of acontact exceeding a first intensity threshold during the input triggersa first response. In some embodiments, the response of the device toinputs detected by the device depends on criteria that include both thecontact intensity during the input and time-based criteria. For example,for some “deep press” inputs, the intensity of a contact exceeding asecond intensity threshold during the input, greater than the firstintensity threshold for a light press, triggers a second response onlyif a delay time has elapsed between meeting the first intensitythreshold and meeting the second intensity threshold. This delay time istypically less than 200 ms in duration (e.g., 40, 100, or 120 ms,depending on the magnitude of the second intensity threshold, with thedelay time increasing as the second intensity threshold increases). Thisdelay time helps to avoid accidental deep press inputs. As anotherexample, for some “deep press” inputs, there is a reduced-sensitivitytime period that occurs after the time at which the first intensitythreshold is met. During the reduced-sensitivity time period, the secondintensity threshold is increased. This temporary increase in the secondintensity threshold also helps to avoid accidental deep press inputs.For other deep press inputs, the response to detection of a deep pressinput does not depend on time-based criteria.

In some embodiments, one or more of the input intensity thresholdsand/or the corresponding outputs vary based on one or more factors, suchas user settings, contact motion, input timing, application running,rate at which the intensity is applied, number of concurrent inputs,user history, environmental factors (e.g., ambient noise), focusselector position, and the like. Exemplary factors are described in U.S.patent application Ser. Nos. 14/399,606 and 14/624,296, which areincorporated by reference herein in their entireties.

For example, FIG. 4C illustrates a dynamic intensity threshold 480 thatchanges over time based in part on the intensity of touch input 476 overtime. Dynamic intensity threshold 480 is a sum of two components, firstcomponent 474 that decays over time after a predefined delay time p1from when touch input 476 is initially detected, and second component478 that trails the intensity of touch input 476 over time. The initialhigh intensity threshold of first component 474 reduces accidentaltriggering of a “deep press” response, while still allowing an immediate“deep press” response if touch input 476 provides sufficient intensity.Second component 478 reduces unintentional triggering of a “deep press”response by gradual intensity fluctuations of a touch input. In someembodiments, when touch input 476 satisfies dynamic intensity threshold480 (e.g., at point 481 in FIG. 4C), the “deep press” response istriggered.

FIG. 4D illustrates another dynamic intensity threshold 486 (e.g.,intensity threshold I_(D)). FIG. 4D also illustrates two other intensitythresholds: a first intensity threshold I_(H) and a second intensitythreshold I_(L). In FIG. 4D, although touch input 484 satisfies thefirst intensity threshold I_(H) and the second intensity threshold I_(L)prior to time p2, no response is provided until delay time p2 haselapsed at time 482. Also in FIG. 4D, dynamic intensity threshold 486decays over time, with the decay starting at time 488 after a predefineddelay time p1 has elapsed from time 482 (when the response associatedwith the second intensity threshold I_(L) was triggered). This type ofdynamic intensity threshold reduces accidental triggering of a responseassociated with the dynamic intensity threshold I_(D) immediately after,or concurrently with, triggering a response associated with a lowerintensity threshold, such as the first intensity threshold I_(H) or thesecond intensity threshold I_(L).

FIG. 4E illustrate yet another dynamic intensity threshold 492 (e.g.,intensity threshold I_(D)). In FIG. 4E, a response associated with theintensity threshold I_(L) is triggered after the delay time p2 haselapsed from when touch input 490 is initially detected. Concurrently,dynamic intensity threshold 492 decays after the predefined delay timep1 has elapsed from when touch input 490 is initially detected. So adecrease in intensity of touch input 490 after triggering the responseassociated with the intensity threshold I_(L), followed by an increasein the intensity of touch input 490, without releasing touch input 490,can trigger a response associated with the intensity threshold I_(D)(e.g., at time 494) even when the intensity of touch input 490 is belowanother intensity threshold, for example, the intensity threshold I_(L).

An increase of characteristic intensity of the contact from an intensitybelow the light press intensity threshold to an intensity between thelight press intensity threshold and the deep press intensity thresholdis sometimes referred to as a “light press” input. An increase ofcharacteristic intensity of the contact from an intensity below the deeppress intensity threshold to an intensity above the deep press intensitythreshold is sometimes referred to as a “deep press” input. An increaseof characteristic intensity of the contact from an intensity below thecontact-detection intensity threshold IT₀ to an intensity between thecontact-detection intensity threshold IT₀ and the light press intensitythreshold IT_(L) is sometimes referred to as detecting the contact onthe touch-surface. A decrease of characteristic intensity of the contactfrom an intensity above the contact-detection intensity threshold IT₀ toan intensity below the contact-detection intensity threshold IT₀ issometimes referred to as detecting liftoff of the contact from thetouch-surface. In some embodiments IT₀ is zero. In some embodiments, IT₀is greater than zero. In some illustrations a shaded circle or oval isused to represent intensity of a contact on the touch-sensitive surface.In some illustrations, a circle or oval without shading is usedrepresent a respective contact on the touch-sensitive surface withoutspecifying the intensity of the respective contact.

In some embodiments, described herein, one or more operations areperformed in response to detecting a gesture that includes a respectivepress input or in response to detecting the respective press inputperformed with a respective contact (or a plurality of contacts), wherethe respective press input is detected based at least in part ondetecting an increase in intensity of the contact (or plurality ofcontacts) above a press-input intensity threshold. In some embodiments,the respective operation is performed in response to detecting theincrease in intensity of the respective contact above the press-inputintensity threshold (e.g., the respective operation is performed on a“down stroke” of the respective press input). In some embodiments, thepress input includes an increase in intensity of the respective contactabove the press-input intensity threshold and a subsequent decrease inintensity of the contact below the press-input intensity threshold, andthe respective operation is performed in response to detecting thesubsequent decrease in intensity of the respective contact below thepress-input threshold (e.g., the respective operation is performed on an“up stroke” of the respective press input).

In some embodiments, the device employs intensity hysteresis to avoidaccidental inputs sometimes termed “jitter,” where the device defines orselects a hysteresis intensity threshold with a predefined relationshipto the press-input intensity threshold (e.g., the hysteresis intensitythreshold is X intensity units lower than the press-input intensitythreshold or the hysteresis intensity threshold is 75%, 90%, or somereasonable proportion of the press-input intensity threshold). Thus, insome embodiments, the press input includes an increase in intensity ofthe respective contact above the press-input intensity threshold and asubsequent decrease in intensity of the contact below the hysteresisintensity threshold that corresponds to the press-input intensitythreshold, and the respective operation is performed in response todetecting the subsequent decrease in intensity of the respective contactbelow the hysteresis intensity threshold (e.g., the respective operationis performed on an “up stroke” of the respective press input).Similarly, in some embodiments, the press input is detected only whenthe device detects an increase in intensity of the contact from anintensity at or below the hysteresis intensity threshold to an intensityat or above the press-input intensity threshold and, optionally, asubsequent decrease in intensity of the contact to an intensity at orbelow the hysteresis intensity, and the respective operation isperformed in response to detecting the press input (e.g., the increasein intensity of the contact or the decrease in intensity of the contact,depending on the circumstances).

For ease of explanation, the description of operations performed inresponse to a press input associated with a press-input intensitythreshold or in response to a gesture including the press input are,optionally, triggered in response to detecting: an increase in intensityof a contact above the press-input intensity threshold, an increase inintensity of a contact from an intensity below the hysteresis intensitythreshold to an intensity above the press-input intensity threshold, adecrease in intensity of the contact below the press-input intensitythreshold, or a decrease in intensity of the contact below thehysteresis intensity threshold corresponding to the press-inputintensity threshold. Additionally, in examples where an operation isdescribed as being performed in response to detecting a decrease inintensity of a contact below the press-input intensity threshold, theoperation is, optionally, performed in response to detecting a decreasein intensity of the contact below a hysteresis intensity thresholdcorresponding to, and lower than, the press-input intensity threshold.As described above, in some embodiments, the triggering of theseresponses also depends on time-based criteria being met (e.g., a delaytime has elapsed between a first intensity threshold being met and asecond intensity threshold being met).

User Interfaces and Associated Processes

Attention is now directed towards embodiments of user interfaces (“UI”)and associated processes that may be implemented on an electronicdevice, such as portable multifunction device 100 or device 300, with adisplay, a touch-sensitive surface, and one or more sensors to detectintensity of contacts with the touch-sensitive surface.

FIGS. 5A-5TT illustrate exemplary user interfaces for processing touchinputs in accordance with some embodiments. The user interfaces in thesefigures are used to illustrate the processes described below, includingmethod 600 described below with respect to FIGS. 6A-6C, method 650described below with respect to FIG. 6D, method 700 described below withrespect to FIGS. 7A-7C, method 750 described below with respect to FIG.7D, method 800 described below with respect to FIGS. 8A-8C, method 900described below with respect to FIGS. 9A-9D, method 1000 described belowwith respect to FIGS. 10A-10D, method 1100 described below with respectto FIGS. 11A-11C, and method 1200 described below with respect to FIGS.12A-12C. Although some of the examples which follow will be given withreference to inputs on a touch-sensitive surface 451 that is separatefrom the display 450, in some embodiments, the device detects inputs ona touch-screen display (where the touch-sensitive surface and thedisplay are combined), as shown in FIG. 4A.

FIG. 5A illustrates window 506 of an application displayed on display450 (e.g., display 340, touch screen 112) of a device (e.g., device 300,100). The application with which window 506 is associated is anapplication that displays content, such as an email application.Content, such as document 508, is displayed in window 506. Document 508optionally includes embedded content, attached content, and/or links toother content (for convenience, collectively referred to below as“embedded content”). The embedded content is, optionally, represented bydocument icons 510. For example, document 508 includes document icons510-1 and 510-2, each of which corresponds to respective embeddedcontent (e.g., a document) embedded in document 508. The respectivedocuments corresponding to icons 510-1 and 510-2 are associated withrespective applications. For example, icon 510-1 optionally correspondsto a presentation document, which is associated with a presentationapplication. The document associated with an icon 510 is, optionally, aword processing document, a spreadsheet, a presentation, a drawing, agraphic or image, an audio file, a video file, a text document, or aPortable Document Format document (sometimes referred to as a PDF).

Cursor 502 is also displayed on display 450. Cursor 502 is an example ofa focus selector. A user may move cursor 502 on display 450 (e.g., usingtouch-sensitive surface 451 of the device) to bring focus to an elementdisplayed on display 450 (e.g., a user interface object, an icon, alink, etc.) by moving a contact on touch-sensitive surface 451. FIG. 5Ashows movement of contact 504 down and to the left on touch-sensitivesurface 451 that causes the device to move cursor 502 down and to theleft on display 504.

FIG. 5A also shows that intensity of contact 504 is below a firstintensity threshold (e.g., “IT₁”). In some embodiments, contact 504 isdeemed to be in stage 0 based on a determination that the intensity ofcontact 504 is below the first intensity threshold. A stage progressvalue indicates a normalized intensity of contact 504, where anintensity that corresponds to the first intensity threshold is deemed tocorrespond to 100% (or 1) stage progress value for stage 0, and anintensity that corresponds to a base intensity threshold or a detectionintensity threshold (e.g., “IT₀”) is deemed to correspond to 0% (or 0)stage progress value for stage 0. When the intensity of contact 504increases from a value below the first intensity threshold andapproaches the first intensity threshold, the stage progress valueincreases toward 100%.

When the intensity of contact 504 increases from an intensity below thefirst intensity threshold to an intensity above the first intensitythreshold, the device expands icon 510-1 to show an animation ofconverting icon 510-1 into preview interface 512 (FIG. 5D). Inaccordance with a determination that the intensity of contact 504satisfies (e.g., is above) the first intensity threshold, contact 504 isdeemed to be in stage 1. The stage progress value is updated forstage 1. In some embodiments, an intensity that corresponds to the firstintensity threshold is deemed to correspond to 0% (or 0) stage progressvalue for stage 1, and an intensity that corresponds to a secondintensity threshold (e.g., “IT₂”) is deemed to correspond to 100% (or 1)stage progress value for stage 1.

FIG. 5B shows that, in response to the detection of increase inintensity of contact 504 above the first intensity threshold, the deviceshows the animation of converting icon 510-1 into preview interface 512(FIG. 5D). In some embodiments, the stage progress value of contact 504is used to initiate the animation (e.g., the animation is initiated inresponse to determining that the stage progress value of contact 504satisfies predefined criteria). In some embodiments, the transitionprogress value of contact 504 is used to initiate the animation (e.g.,the animation is initiated in response to determining that thetransition progress value of contact 504 satisfies predefined criteria)(not shown).

FIG. 5C shows that, in response to the detection of further increase inintensity of contact 504, the device shows a further expanded icon 510.Contact 504 is still deemed to be in stage 1, and the stage progressvalue has increased further.

FIG. 5C also shows that a transition progress value has increased from0%. The transition progress value indicates that contact 504 is within apredefined intensity range from an intensity threshold (e.g., the secondintensity threshold). In some cases, the transition progress value alsoindicates how much additional intensity is required for a stagetransition (e.g., from stage 1 to stage 2) in normalized intensity. Insome embodiments, the intensity range is defined by a transition rangeintensity threshold (also called herein a transition intensitythreshold) that indicates one end of the intensity range and anactivation intensity threshold that indicates an intensity at which astage transition occurs. In FIG. 5C, the transition intensity thresholdfor a transition from stage 1 to stage 2 is identical to a releaseintensity threshold “IT_(2R)” for stage 2. However, the transitionintensity threshold need not be identical to a release intensitythreshold.

FIG. 5D shows that the intensity of contact 504 has increased above thesecond intensity threshold (e.g., “IT₂”). In accordance with adetermination that the intensity of contact 504 satisfies (e.g., isabove) the second intensity threshold, contact 504 is deemed to be instage 2. In response, preview interface 512 is displayed. The stageprogress value for contact 504 is updated for stage 2. In someembodiments, an intensity that corresponds to the second intensitythreshold is deemed to correspond to 0% (or 0) stage progress value forstage 2, and an intensity that corresponds to a third intensitythreshold (e.g., “IT₃”) is deemed to correspond to 100% (or 1) stageprogress value for stage 2.

FIG. 5E shows further increase in the intensity of contact 504. Contact504 has not satisfied the third intensity threshold, and still remainsin stage 2. The stage progress value and the transition progress valueare increased.

FIG. 5F shows that the intensity of contact 504 has decreased below thesecond intensity threshold (e.g., “IT₂”), but remains above a releaseintensity threshold for stage 2 (e.g., “IT_(2R)”). In accordance with adetermination that the intensity of contact 504 does not satisfy (e.g.,remains above) the release intensity threshold for stage 2, contact 504is deemed to remain in stage 2. Thus, preview interface 512 remains ondisplay. In FIG. 5F, the stage progress value is 0% for stage 2, becausethe intensity of contact 504 is below the second intensity threshold.The transition progress value indicates that the intensity of contact504 is within a predefined intensity range from the release intensitythreshold for stage 2.

FIG. 5G shows that the intensity of contact 504 has further decreasedbelow the release intensity threshold for stage 2. In accordance with adetermination that the intensity of contact 504 satisfies (e.g., isbelow) the release intensity threshold for stage 2, contact 504 isdeemed to be in stage 1. Preview interface 512 (FIG. 5F) ceased to bedisplayed and an animation that shows a transition from icon 510-1 topreview interface 512 is shown in accordance with the intensity ofcontact 504.

FIG. 5H shows that, in some embodiments, a release intensity thresholdis identical to an activation intensity threshold (e.g.,“IT₁”=“IT_(1R)”). Thus, the release intensity threshold is notseparately shown in FIG. 5H.

FIG. 5I shows that the intensity of contact 504 has significantlyincreased to satisfy the third intensity threshold (e.g., “IT₃”). Inaccordance with a determination that the intensity of contact 504satisfies the third intensity threshold, contact 504 is deemed to be instage 3. In response, preview interface 512 is enlarged to fill window506.

FIG. 5J shows that intensity thresholds have been changed. For example,in some embodiments, the device determines that previously usedintensity thresholds are too low for a user (e.g., the user tends toapply high intensity contacts frequently or is found to have difficultyapplying low intensity contacts), and updates the intensity thresholds(e.g., increases at least one of the intensity thresholds and/ordecreases at least one of the intensity thresholds). In someembodiments, a software application (e.g., a user interface application,such as an email application) determines that the intensity thresholdsbased on user interactions and/or internal operations (e.g., so as toavoid conflicts with other functions, such as accessibility). In someembodiments, the intensity thresholds are associated with a particularintensity model. Thus, the device switches from a first intensity model(e.g., “normal user” intensity model) to a second intensity model (e.g.,“high intensity user” intensity model) that is distinct from the firstintensity model to use different intensity thresholds.

In FIG. 5J, although the intensity of contact 504 has not changed fromFigure SI, due to the change in the intensity thresholds, contact 504 isdeemed to be in stage 2. Thus, preview interface 512 ceases to fillwindow 506 (e.g., preview interface 512 is reduced from the enlargedpreview interface 512 shown in FIG. 5I).

FIG. 5K shows that the intensity of contact 504 has decreased from abovethe release intensity threshold for stage 2 (“IT_(2R-2)”) to below therelease intensity threshold for stage 2. In accordance with adetermination that the intensity of contact 504 satisfies (e.g., isbelow) the release intensity threshold for stage 2, contact 504 isdeemed to be in stage 1. In response, preview interface 512 is replacedwith an animation showing a transition between preview interface 512 andicon 510-1.

FIG. 5L shows that the intensity of contact 504 has further decreased tobelow a previous release intensity threshold for stage 1 (e.g.,“IT_(1R)”) and above a current release intensity threshold for stage 1(e.g., “IT_(1R-2)”). In accordance with a determination that theintensity of contact 504 does not satisfy (e.g., remains above) thecurrent release intensity threshold for stage 1, contact 1 is deemed tobe in stage 1. If the previous release intensity threshold was to beused, contact 504 would be deemed to be in stage 0 in accordance with adetermination that the intensity of contact 504 satisfies the previousrelease intensity threshold for stage 1.

FIG. 5M shows that contact 504 ceases to be detected on touch-sensitivesurface 451. In FIG. 5M, the intensity thresholds are reset so thatprevious intensity thresholds are used. For example, in someembodiments, changes to the intensity thresholds expire upon lift-off ofa contact from touch-sensitive surface 451 (or alternatively theintensity of the contact falling below the detection intensity thresholdIT₀).

FIGS. 5N-5O illustrate that subsequent contact 514 is detected ontouch-sensitive surface 451, and contact 514 is processed in accordancewith the intensity thresholds that have been reset.

FIGS. 5P-5R illustrate that the intensity thresholds are changed whileno contact is detected on touch-sensitive surface 451 (or prior todetecting contact 516). FIGS. 5P-5R also illustrate that contact 516 isprocessed in accordance with the changed intensity thresholds.

In some embodiments, the intensity thresholds are predefined orpreselected for respective regions. For example, in FIG. 5S, icon 510-1is associated with a first intensity model that includes a first set ofintensity thresholds, and an email icon (e.g., 546-1 through 546-7) isassociated with a second intensity model that is distinct from the firstintensity model and includes a second set of intensity thresholds. FIG.5S also shows that at least a portion of a user interface 518 of asecond software application (e.g., a drawing application) is also shown.In FIG. 5S, the user interface 518 of the second software application isassociated with a third intensity model that is distinct from the firstintensity model and the second intensity model and includes a third setof intensity threshold.

FIG. 5T shows that at least a portion of a user interface of a firstsoftware application (e.g., an email application) that includes icons510-1 and 510-2 and at least a portion of a user interface 518 of asecond software application (e.g., a drawing application) areconcurrently displayed.

FIG. 5T also illustrates a movement of contact 520 from a first location520-A on touch-sensitive surface 451 across touch-sensitive surface 451(e.g., while remaining contact on touch-sensitive surface 451) to asecond location 520-B on touch-sensitive surface 451. FIG. 5T furtherillustrate a corresponding movement of cursor 502 from a first location502-A on display 450 to a second location 502-B on display 450. In FIG.5T, the first location 520-A on touch-sensitive surface 451 correspondsto the first location 502-A on display 450 and the second location 520-Bon touch-sensitive surface 451 corresponds to the second location 502-Bon display 450.

FIG. 5U illustrates a movement of contact 520 from the second location520-B on touch-sensitive surface 451 across touch-sensitive surface 451to the first location 520-A on touch-sensitive surface 451 and acorresponding movement of cursor 502 from the second location 502-B ondisplay 450 to the first location 502-A on display 450.

FIGS. 5V-5W show that subsequent contact 522 is detected at the firstlocation 522-A and intensity of contact 522 is increased. In FIGS.5V-5W, contact 522 is processed in accordance with a first intensitymodel that includes a first set of intensity thresholds (e.g., anintensity model associated with icon 510-1 or the first location 502-Aon the display).

FIG. 5X shows that the intensity of contact 522 is reduced (e.g., belowany activation intensity threshold for the first location 502-A on thedisplay) and moved across touch-sensitive surface 451 to the secondlocation 522-B on touch-sensitive surface 451. FIG. 5X also shows acorresponding movement of cursor 502 from the first location 502-A ondisplay 451 to the second location 502-B on display 450.

FIGS. 5Y-5Z show that contact 524 is detected at the second location522-B on touch-sensitive surface 451 and intensity of contact 524 isincreased. In FIGS. 5Y-5Z, contact 524 is processed in accordance with asecond intensity model that is distinct from the first intensity modeland includes a second set of intensity thresholds (e.g., an intensitymodel associated with user interface 518 of the second softwareapplication or the second location 502-B on the display).

FIG. 5AA illustrates that the intensity of contact 524 is reduced (e.g.,below any activation intensity threshold for the second location 502-Bon the display) and moved across touch-sensitive surface 451 to a thirdlocation 524-C on touch-sensitive surface 451. FIG. 5AA also shows acorresponding movement of cursor 502 from the second location 502-B ondisplay 451 to the third location 502-C on display 450. The thirdlocation 502-C is deemed to correspond to an overlapping region of theuser interface of the first software application and the user interface518 of the second software application.

FIG. 5BB illustrates that contact 526 is detected at the third location526-C on touch-sensitive surface 451. In some embodiments, contact 526is processed in accordance with the second intensity model (e.g.,associated with user interface 518, which is the topmost foreground userinterface in FIG. 5BB). However, in some other embodiments, contact 526is processed in accordance with the first intensity model (even thoughicon 510-2 is not displayed on display 450) based on the priority of thefirst intensity model and the priority of the second intensity model(e.g., the first intensity model is selected because the first intensitymodel has a higher priority than the second intensity model). In someembodiments or in some circumstances, the user interface of the firstsoftware application is displayed over user interface 518 as shown inFIG. 5CC.

FIG. 5DD illustrates that contact 528 is detected at the third location528-C on touch-sensitive surface 451. In some embodiments, contact 528is processed in accordance with the first intensity model (e.g.,associated with user interface 518, which is the topmost foreground userinterface in FIG. 5DD). However, in some other embodiments, contact 528is processed in accordance with the second intensity model (even thoughthe user interface 518 of the second software application is overlaid bythe user interface of the first software application at the thirdlocation 502-C on display) based on the priority of the first intensitymodel and the priority of the second intensity model. Although thepriority of the first intensity model was higher than the priority ofthe second intensity model during the operations illustrated in FIGS.5BB-5CC, the priorities may be updated real-time. For example, the firstsoftware application and/or the second software application may send arequest to process any contact in a particular display or user interfaceregion, which in this example includes the third location 528-C, inaccordance with the second intensity model, and as a result, the contactat the third location 528-C is processed (e.g., by contact motion module130, FIG. 3G) using the second intensity model.

FIGS. 5FF-5JJ illustrate user interfaces associated with handwritingrecognition (e.g., handwritten characters, hand-drawn shapes, etc.) inaccordance with some embodiments.

FIG. 5FF shows a user interface of a word processing softwareapplication partially overlaid by handwriting input tool region 540(e.g., a character input tool region). As shown in FIGS. 5FF-5JJ,handwriting input tool region 540 includes a plurality of selectionregions (e.g., at least some of 532-1 through 532-8) and handwritinginput region 530 (e.g., a character input region). A respectiveselection region (e.g., 532-1) on display 450 corresponds to a region(e.g., 534-1) on touch-sensitive surface 451. In addition, handwritinginput region 530 on display 450 corresponds region 542 ontouch-sensitive surface 451. In some embodiments, the respectiveselection region (e.g., 532-1) on display 450 is associated with anintensity model that defines two or more intensity stages (e.g., a stateindicating that a contact is detected but a corresponding user interfaceelement has not been activated, and a state indicating that a contact isdetected and the corresponding user interface element has beenactivated). In some embodiments, tactile outputs are enabled for theintensity model associated with the respective selection region. Forexample, a press input (with sufficient intensity) will trigger atactile output, which indicates that intensity of the press input issufficient to activate a corresponding user interface element.

FIG. 5FF also shows that contact 536 is detected in region 542 oftouch-sensitive surface 451 and moved across touch-sensitive surface 451along a particular path. In some embodiments, region 542 oftouch-sensitive surface 451 is associated with an intensity model thathas a single stage for which tactile outputs are suppressed. Thisprevents tactile outputs from interfering with a user's writing/drawingon the touch-sensitive surface. In some embodiments, for a contact thatis initially detected on region 542 of touch-sensitive surface 451,tactile outputs are suppressed even if the contact moves acrosstouch-sensitive surface 451 to a region (e.g., 534-1) that correspondsto a respective selection region (e.g., 532-1). This “latching” featureis described further below with respect to FIGS. 5KK-5TT.

In some embodiments, for a contact starting from the region 542, tactileoutputs are suppressed even when the contact moves to a location thatcorresponds to a selection region on display.

FIG. 5GG shows that graphical element 544 that corresponds to theparticular path of contact 536 is displayed in handwriting input region530. In some embodiments, a width of graphical element 544 varies alongits path based on one or more measured characteristics of contact 536 inregion 542 of touch-sensitive surface 451 while contact 536 follows theparticular path. In some embodiments, the width is determined based onintensity and/or a speed of contact 536. In some embodiments, the widthis increased when the intensity of contact 536 increases. In someembodiments, the width is reduced when the speed of contact 536increases.

FIG. 5GG also shows that a plurality of selection regions 532-5 through532-8 is displayed. Each of selection regions 532-5 through 532-8includes a character that is selected based on graphical element 544.For example, characters in selection regions 532-5 through 532-8 areselected in accordance with (e.g., that best match) the shape ofgraphical element 544 (e.g., that best match one or more characteristicsof the shape of graphical element 544).

FIG. 5HH shows that contact 538 is detected on region 534-8 oftouch-sensitive surface 451, which corresponds to selection region532-8. Contact 538 is processed using an intensity model that isassociated with region 534-8. For example, a tactile output is generatedwhen intensity of contact 538 crosses one or more thresholds of theintensity model associated with region 534-8. In FIG. 5HH, a charactershown in selection region 532-8 is displayed in the user interface ofthe word processing software application (and inserted into a documentof the word processing software application).

FIG. 5II shows the path of contact 536 on touch-sensitive surface 451,discussed above with respect to FIG. 5FF. In FIG. 5II, shown to theright side of touch-sensitive surface 451 is an intensity graph thatillustrates intensity of contact 536 over time while contact 536 followsthe path on touch-sensitive surface 451. The graph shows that theintensity of contact 536 starts from below a drawing intensity thresholdIT_(D), increases above the drawing intensity threshold, and falls belowthe drawing intensity threshold. In some embodiments, once the intensityof contact 536 satisfies (is above) the drawing intensity threshold,contact 536 continues to be processed even if the intensity of contact536 falls below the drawing intensity threshold. For example, a portionof the path in which the contact has intensity below the drawingintensity threshold is used to extend or update the graphical element544. However, in some embodiments, the intensity of contact 536 needs tosatisfy the drawing intensity threshold to initiate display of agraphical element.

FIG. 5JJ shows that contact 536 ceases to be detected while followingthe path on touch-sensitive surface 451. FIG. 5JJ also shows thatcontact 538 is detected on touch-sensitive surface 451, continuing thepath. For example, contact 536 may be slightly lifted-off while drawingthe path on touch-sensitive surface 451 and quickly brought back intocontact with touch-sensitive surface 451 to complete the path. In someembodiments, a brief absence of contact with touch-sensitive surface 451is ignored and the path of contact 536 and the path of contact 544 aremerged to form a single continuous path. In some embodiments, the mergerof the two paths is performed based on a time interval between thelift-off of contact 536 and the detection of contact 544 satisfyingpredefined timing criteria (e.g., less than 0.1 second, 0.2, second,0.5, second, etc.). In some other embodiments, the path of contact 536and the path of contact 544 are maintained separately. In someembodiments, the path of contact 544 is initiated in response todetermining that intensity of contact 544 satisfies the drawingintensity threshold (e.g., instead of contact 544 merely contactingtouch-sensitive surface 451).

Although FIGS. 5FF-5JJ are described above with respect to recognitionof handwritten characters, similar methods, devices, and user interfacesmay be used for recognition of hand drawn content (sometimes called orincluding handwritten content) of a different type (e.g., recognition ofshapes or a combination of shapes and characters/letters).

In addition, although FIGS. 5FF-5JJ illustrate a user interface of ahandwriting input tool region with a plurality of selection regions, insome embodiments, a drawing region without any selection regions isused. For example, when character recognition (or shape recognition) isnot needed, the plurality of selection regions may be omitted. In someembodiments, a single intensity model is used for the entire handwritinginput tool region.

FIGS. 5KK-5TT illustrate latching of touch inputs in accordance withsome embodiments.

FIG. 5KK shows a window 506 that includes a user interface of a softwareapplication (e.g., an email application). The user interface in window506 includes a plurality of regions (e.g., email regions 546-1 through546-7, icons 510-1, and 510-2). Some of these regions are associatedwith different intensity models. For example, icon 510-1 is associatedwith a first intensity model and region 546-7 is associated with asecond intensity model that is distinct from the first intensity model.

FIG. 5LL shows contact 548 is detected at location 548-A ontouch-sensitive surface 451 that corresponds to icon 510-1 on display450. In FIG. 5LL, icon 510-1 is visually distinguished to indicate thedetection of contact 548 on touch-sensitive surface 451.

FIGS. 5MM-5PP show operations associated with a movement of a contactwithout latching to a region of a user interface in accordance with someembodiments.

FIG. 5MM shows that intensity of contact 548 has increased and icon510-1 is further visually distinguished to indicate increase in theintensity of contact 548.

FIG. 5NN shows that the intensity of contact 548 has been reduced andicon 510-1 is visually distinguished to indicate decrease in theintensity of contact 548. FIG. 5NN also shows that a movement of contact548 across touch-sensitive surface 451 to location 548-B ontouch-sensitive surface 451, and a corresponding movement of cursor 502to region 546-7 on display 450. Prior to the movement of contact 548 tolocation 548-B on touch-sensitive surface 451, the intensity of contact548 has not satisfied a latching intensity threshold (e.g., “IT_(L)”)for icon 510-1.

FIG. 5OO shows that, in response to the movement of cursor 502 to region546-7 on display 450, visual distinction of icon 510-1 is removed andregion 546-7 is visually distinguished to indicate detection of contact548 at a location 548-B that correspond to region 546-7 on display 450.

FIG. 5PP shows that the intensity of contact 548 has increased above theactivation intensity threshold (e.g., “IT_(A)”) for region 546-7. Inresponse, region 546-7 is visually distinguished to indicate that region546-7 has been activated. In addition, the visual distinction of region546-2, which was previously activated, is removed. FIG. 5PP also showsthat window 506 is updated to show content of Email 7 that correspondsto region 546-7.

FIGS. 5QQ-5TT show operations associated with a movement of a contactwhile latched to a region of a user interface in accordance with someembodiments.

FIG. 5QQ shows that the intensity of contact 548 has increased tosatisfy the latching intensity threshold (e.g., “IT_(L)”) for icon510-1. FIG. 5QQ also illustrates that icon 510-1 is further visuallydistinguished to indicate that cursor 502 is latched to icon 510-1.

FIG. 5RR shows a movement of contact 548 across touch-sensitive surface451 to location 548-B on touch-sensitive surface 451 while maintainingintensity above the latching intensity threshold (e.g., “IT_(L)”) foricon 510-1, and a corresponding movement of cursor 502 to region 546-7on display 450. In FIG. 5RR, although the intensity of contact 548 wouldsatisfy the activation intensity threshold (e.g., “IT_(A)”) for region546-7, region 546-7 is not activated because cursor 502 is latched toicon 510-1.

FIG. 5SS shows that the intensity of contact 548 has further increasedto satisfy an activation intensity threshold (e.g., “IT_(A)”) for icon510-1, which activates icon 510-1 even though cursor 502 is located inregion 546-7.

FIG. 5TT shows that, in response to the intensity of contact 548satisfying the activation intensity threshold (e.g., “IT_(A)”) for icon510-1, icon 510-1 is activated. In FIG. 5TT, preview interface 512 isdisplayed in response to the intensity of contact 548 satisfying theactivation intensity threshold (e.g., “IT_(A)”) for icon 510-1.

FIGS. 6A-6C illustrate a flow diagram of a method 600 of processing atouch input based on an intensity stage of the touch input in accordancewith some embodiments. The method 600 is performed at an electronicdevice (e.g., device 300, FIG. 3A or 3B, or portable multifunctiondevice 100, FIG. 1A) with a touch-sensitive surface, and one or moresensors to detect intensity of contacts with the touch-sensitivesurface. In some embodiments, the electronic device includes a display.In some embodiments, the display is a touch-screen display and thetouch-sensitive surface is on or integrated with the display. In someembodiments, the display is separate from the touch-sensitive surface.Some operations in method 600 are, optionally, combined and/or the orderof some operations is, optionally, changed.

As described below, the method 600 simplifies processing of touchinputs. The method reduces the computational burden on a softwareapplication, thereby creating a more efficient electronic device. Inaddition, the size of the software application is reduced, therebyoccupying less storage space and memory.

The device detects (602) a touch input on the touch-sensitive surface(e.g., contact 504, FIG. 5B).

In response to detecting the touch input on the touch-sensitive surface,the device determines (604) an intensity of the touch input (e.g., aforce applied by the touch input) on the touch-sensitive surface (e.g.,Intensity of Contact 504, FIG. 5B).

In accordance with the intensity of the touch input on thetouch-sensitive surface and one or more preselected intensitythresholds, the device determines (606) an intensity stage of the touchinput. For example, contact 504 is determined to be in stage 1 in FIG.5B. The intensity stage of the touch input is selected from a pluralityof predefined intensity stages (e.g., stages 1, 2, and 3). In someembodiments, the plurality of predefined intensity stages includes threeor more distinct intensity stages. In some embodiments, the plurality ofpredefined intensity stages includes three or more non-overlappingintensity stages.

In some embodiments, determining the intensity stage of the touch inputincludes comparing the intensity of the touch input with the one or morepreselected intensity thresholds. In some embodiments, determining theintensity stage of the touch input depends on a previous intensity stageof the touch input. For example, if the touch input has been in stage 2,the intensity of the touch input is compared with an activationintensity threshold for stage 3 to determine whether the touch inputtransitions to stage 3, and with a release intensity threshold for stage2 to determine whether the touch input transitions to stage 1. In someembodiments, the determination of the intensity stage of the touch inputis performed by the contact motion module 130 (FIG. 3G) (e.g., thecontact intensity module 175 in the contact motion module 130).

In some embodiments, the intensity stage of the touch input isdetermined in accordance with the intensity of the touch input on thetouch-sensitive surface, the one or more predefined intensitythresholds, and time-based criteria described above. As explained above,for some “deep press” inputs, the intensity of a contact exceeding asecond intensity threshold during the input, greater than the firstintensity threshold for a light press, triggers a second response onlyif a delay time has elapsed between meeting the first intensitythreshold and meeting the second intensity threshold.

In some embodiments, the device stores (608) a first softwareapplication (e.g., E-mail Client Module 140, FIG. 3A). The intensitystage of the touch input is determined by a contact intensity modulethat is distinct and separate from the first software application (e.g.,a firmware embedded in a touch-sensitive input device, such as atrackpad, that include the touch-sensitive surface, an operating system,etc.). For example, in FIG. 3G, contact intensity module 175 incontact/motion module 130 sends touch information 472, which in someembodiments include intensity stage 478, to application 136.

In some embodiments, the device sends (610) to the first softwareapplication from the contact intensity module touch information thatidentifies the intensity stage of the touch input (e.g., in FIG. 3G,touch information 472 including intensity stage 478 is sent from contactintensity module 175 to application 136).

In some embodiments, the device sends (612, FIG. 6B) information fromthe contact intensity module, the information indicating that theintensity of the touch input is available to the first softwareapplication (e.g., availability information 474, FIG. 3G).

In some embodiments, the device repeats (614) the operations ofdetermining an intensity of the touch input, determining an intensitystage of the touch input, and sends touch information while the touchinput is detected on the touch-sensitive surface. In some embodiments,the operations are repeated at a predefined interval (or a predefinedfrequency). For example, the device repeats such operations while acontact remains on the touch-sensitive surface.

In some embodiments, the device determines (616) one or moreintensity-based progress values of the touch input based on an intensityrange associated with the determined intensity stage; and sends touchinformation to the first software application (e.g., touch information472 in FIG. 3G). The touch information includes the one or moreintensity-based progress values of the touch input (e.g., stage progressvalue 482 and transition progress value 484) and information identifyingthe intensity stage of the touch input (e.g., intensity stage 478).

In some embodiments, the one or more intensity-based progress values ofthe touch input include (618) a transition progress value of the touchinput (e.g., transition progress value 484, FIG. 3G). In someembodiments, the transition progress value indicates a normalizedintensity required for a transition to an intensity stage that isadjacent to the determined intensity stage.

In some embodiments, the one or more intensity-based progress values ofthe touch input include (620) a stage progress value of the touch input(e.g., stage progress value 482, FIG. 3G). In some embodiments, thestage progress value indicates a normalized intensity that is based onthe intensity of the touch input and predefined intensity thresholds forthe determined intensity stage.

In some embodiments, in response to detecting the touch input, thedevice determines (622) a first intensity applied by the touch input onthe touch-sensitive surface (e.g., intensity of contact 504 in FIG. 5C).In accordance with a determination that the first intensity applied bythe touch input on the touch-sensitive surface does not satisfy a stageactivation intensity threshold for a second intensity stage, the devicedetermines that the touch input is in a first intensity stage that isdistinct from the second intensity stage (e.g., in FIG. 5C, contact 504remains in stage 1). In some embodiments, the second intensity stage isadjacent to the first intensity stage. In some embodiments, the methodincludes sending first touch information that identifies the firstintensity stage of the touch input to the first software application.Subsequent to determining that the touch input is in the first intensitystage, the device determines a second intensity applied by the touchinput on the touch-sensitive surface (e.g., intensity of contact 504 inFIG. 5D). The second intensity is distinct from the first intensity. Insome embodiments, the device determines that an intensity applied by thetouch input on the touch-sensitive surface has changed from the firstintensity to the second intensity. In accordance with a determinationthat the second intensity applied by the touch input on thetouch-sensitive surface satisfies the stage activation intensitythreshold for the second intensity stage (e.g., intensity of contact 504is above “IT₂” in FIG. 5D), the device determines that the touch inputis in the second intensity stage (e.g., in FIG. 5D, contact 504 is instage 2).

In some embodiments, subsequent to determining that the touch input isin the second intensity stage, the device determines (624, FIG. 6C) athird intensity applied by the touch input on the touch-sensitivesurface (e.g., intensity of contact 504 in FIG. 5F). The third intensityis distinct from the second intensity. In accordance with adetermination that the third intensity does not satisfy a stage releaseintensity threshold for the second intensity stage (e.g., intensity ofcontact 504 is above “IT_(2R)” in FIG. 5F), distinct from the stageactivation intensity threshold for the second intensity stage (e.g.,“IT₂” in FIG. 5F), the device determines that the touch input remains inthe second intensity stage (e.g., contact 504 remains in stage 2 in FIG.5F). Subsequent to determining that the touch input remains in thesecond intensity stage, the device determines a fourth intensity appliedby the touch input on the touch-sensitive surface (e.g., intensity ofcontact 504 in FIG. 5G). The fourth intensity is distinct from the thirdintensity. In accordance with a determination that the fourth intensitysatisfies the stage release intensity threshold for the second intensitystage (e.g., “IT_(2R)” in FIG. 5F), the device determines that the touchinput is in the first intensity stage (e.g., contact 504 is in stage 1in FIG. 5G).

In some embodiments, subsequent to determining that the touch inputremains in the second intensity stage, the device determines (626) athird intensity applied by the touch input on the touch-sensitivesurface (e.g., intensity of contact 504 in FIG. 5I). The third intensityis distinct from the second intensity. In accordance with adetermination that the third intensity satisfies a stage activationthreshold for the third intensity stage (e.g., “IT₃” in FIG. 5I),distinct from the stage activation intensity threshold for the secondintensity stage, the device determines that the touch input is in thethird intensity stage (e.g., contact 504 is in stage 3 in FIG. 5I).

In some embodiments, subsequent to determining that the touch input isin the third intensity stage, the device determines (628) a fourthintensity applied by the touch input on the touch-sensitive surface(e.g., intensity of contact 504 in FIG. 5F). The fourth intensity isdistinct from the third intensity. In accordance with a determinationthat the fourth intensity satisfies a stage release intensity thresholdfor the third intensity stage (e.g., intensity of contact 504 is below“IT_(3R)” in FIG. 5F), distinct from the stage activation threshold forthe third intensity stage, the device determines that the touch input isin the second intensity stage.

In some embodiments, the device identifies (630, FIG. 6A) an intensitymodel identifier from a plurality of predefined intensity modelidentifiers (e.g., exemplary intensity models and correspondingintensity model identifiers described in Appendix A). The intensitystage of the touch input is selected from a plurality of intensitystages that correspond to the identified intensity model identifier.

The device processes (632) the touch input based on the intensity stageof the touch input (e.g., generates a tactile output).

It should be understood that the particular order in which theoperations in FIGS. 6A-6C have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 650, 700, 750, 800, 900, 1000, 1100, and 1200) are alsoapplicable in an analogous manner to method 600 described above withrespect to FIGS. 6A-6C. For example, the touch input processingdescribed above with reference to method 600 optionally have one or moreof the characteristics of the touch input processing based on a touchcharacterization parameter described herein with reference to othermethods described herein (e.g., methods 650, 700, 750, 800, 900, 1000,1100, and 1200). For brevity, these details are not repeated here.

In addition, one of ordinary skill in the art would recognize that manymodifications and variations are possible in view of the aboveteachings.

For example, in accordance with some embodiments, a method is performedat an electronic device with a touch-sensitive surface. The electronicdevice includes one or more sensors to detect intensity of contacts withthe touch-sensitive surface (e.g., the touch-sensitive surface includesone or more sensors to detect the intensity of contacts with thetouch-sensitive surface). In some embodiments, the touch-sensitivesurface is integrated with a display. In some embodiments, thetouch-sensitive surface is separate from the display. The methodincludes, in response detecting the touch input on the touch-sensitivesurface, determining an intensity of the touch input on thetouch-sensitive surface; and, in accordance with the intensity of thetouch input on the touch-sensitive surface and one or more preselectedintensity thresholds, a transition progress value of the touch input.The method also includes processing the touch input based on thetransition progress value of the touch input.

For another example, in accordance with some embodiments, a method isperformed at an electronic device with a touch-sensitive surface. Theelectronic device includes one or more sensors to detect intensity ofcontacts with the touch-sensitive surface (e.g., the touch-sensitivesurface includes one or more sensors to detect the intensity of contactswith the touch-sensitive surface). In some embodiments, thetouch-sensitive surface is integrated with a display. In someembodiments, the touch-sensitive surface is separate from the display.The method includes, in response detecting the touch input on thetouch-sensitive surface, determining an intensity of the touch input onthe touch-sensitive surface; and, in accordance with the intensity ofthe touch input on the touch-sensitive surface and one or morepreselected intensity thresholds, a stage progress value of the touchinput. The method also includes processing the touch input based on thestage progress value of the touch input.

FIG. 6D illustrate a flow diagram of a method 650 of updating a userinterface based on an intensity stage of a touch input in accordancewith some embodiments. The method 650 is performed at an electronicdevice (e.g., device 300, FIG. 3A or 3B, or portable multifunctiondevice 100, FIG. 1A) with a display, a touch-sensitive surface, and oneor more sensors to detect intensity of contacts with the touch-sensitivesurface. In some embodiments, the display is a touch-screen display andthe touch-sensitive surface is on or integrated with the display. Insome embodiments, the display is separate from the touch-sensitivesurface. Some operations in method 650 are, optionally, combined and/orthe order of some operations is, optionally, changed.

The device displays (652) a user interface on the display (e.g., a userinterface of an email application is displayed in FIG. 5A).

While displaying the user interface, the device receives (654) at thefirst software application from a contact intensity module distinct fromthe first software application touch information that identifies anintensity stage of a touch input detected on the touch-sensitive surface(e.g., application 136 receives touch information 472 from contactintensity module 175).

The device updates (656) the user interface in accordance with at leastthe intensity stage of the touch input (e.g., an animation is shown inFIG. 5B).

In some embodiments, the device displays (658) an animation,corresponding to a transition to or from a predefined intensity state ofthe touch input, the animation corresponding to an intensity-basedprogress value of the touch input. For example, the animationillustrated in FIGS. 5A-5D corresponds to a stage progress value ofcontact 504. In some embodiments, the touch information received at thefirst software application from the contact intensity module alsoidentifies the intensity-based progress value of the touch input.

It should be understood that the particular order in which theoperations in FIG. 6D have been described is merely exemplary and is notintended to indicate that the described order is the only order in whichthe operations could be performed. One of ordinary skill in the artwould recognize various ways to reorder the operations described herein.Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 600, 700, 750, 800, 900, 1000, 1100, and 1200) are alsoapplicable in an analogous manner to method 650 described above withrespect to FIG. 6D. For example, the intensity stage described abovewith reference to method 650 optionally have one or more of thecharacteristics of the intensity stage described herein with referenceto other methods described herein (e.g., methods 600, 700, 800, 900,1000, 1100, and 1200). For brevity, these details are not repeated here.

FIGS. 7A-7C illustrate a flow diagram of a method 700 of processingtouch inputs based on intensity model identifiers in accordance withsome embodiments. The method 700 is performed at an electronic device(e.g., device 300, FIG. 3A or 3B, or portable multifunction device 100,FIG. 1A) with a touch-sensitive surface, and one or more sensors todetect intensity of contacts with the touch-sensitive surface. Theelectronic device stores a first software application. In someembodiments, the electronic device includes a display. In someembodiments, the display is a touch-screen display and thetouch-sensitive surface is on or integrated with the display. In someembodiments, the display is separate from the touch-sensitive surface.Some operations in method 700 are, optionally, combined and/or the orderof some operations is, optionally, changed.

As described below, the method 700 simplifies processing of touchinputs. The method reduces the computational burden on a softwareapplication, thereby creating a more efficient electronic device. Inaddition, the size of the software application is reduced, therebyoccupying less storage space and memory.

The device detects (702) a first touch input on the touch-sensitivesurface (e.g., contact 504 in FIG. 5A).

In response to detecting the first touch input on the touch-sensitivesurface, the device determines (704) a first intensity applied by thefirst touch input on the touch-sensitive surface (e.g., intensity ofcontact 504 in FIG. 5A).

The device identifies (706) a first intensity model identifier from aplurality of predefined intensity model identifiers.

In some embodiments, identifying the first intensity model identifierincludes (708) identifying the first software application ascorresponding to the touch input and identifying an intensity modelidentifier registered by (or for or corresponding to) the first softwareapplication as the first intensity model identifier. For example, thedevice may identify use info 458 in intensity models 454 (FIG. 3G) andidentify an intensity model identifier registered by the first softwareapplication.

In some embodiments, in response to detecting the first touch input onthe touch-sensitive surface, the device generates (710) a tactile outputin accordance with the intensity applied by the first touch input on thetouch-sensitive surface and the one or more thresholds associated withthe first intensity model identifier (e.g., a tactile output isgenerated when intensity of a contact crosses one of the intensitythresholds). In some embodiments, a tactile output can be generated byeither the contact intensity module or the first software application(e.g., the contact intensity and/or the first software application maysend instructions to one or more tactile output generators to initiatethe generation of a tactile output).

In accordance with the first intensity applied by the first touch inputon the touch-sensitive surface and one or more thresholds associatedwith the first intensity model identifier, the device determines (712) afirst touch characterization parameter (e.g., intensity stage,transition progress or stage progress). In some embodiments, theintensity thresholds are adjusted based on historical user actions. Forexample, if a user repeatedly applying high intensity that exceedscertain criteria, intensity thresholds are increased for the user.

In some embodiments, the device selects (714) a set of thresholds (e.g.,intensity thresholds), from a plurality of sets of thresholds, inaccordance with the identified intensity model identifier, anddetermines the touch characterization parameter in accordance with theselected set of thresholds. For example, the touch characterizationparameter may be determined using the set of thresholds (e.g.,associated with a first intensity model identifier) shown in FIG. 5J orthe set of thresholds (e.g., associated with a second intensity modelidentifier) shown in FIG. 5K.

In some embodiments, the device identifies (716) one or more (oralternatively two or more) intensity ranges in accordance with theidentified intensity model identifier, and determines a touchcharacterization parameter in accordance with the one or more identifiedintensity ranges. For example, intensity ranges may be used instead of,or in addition to, using intensity thresholds to determine one or moretouch characterization parameters.

Subsequent to determining the first touch characterization parameter,the device sends (718, FIG. 7B) first touch information to the firstsoftware application. The first touch information includes the firstintensity model identifier and the first touch characterizationparameter (e.g., in FIG. 3G, touch information 472 includes intensitymodel identifier 485 and characterization parameters 476).

In some embodiments, sends (720) a stream of intensity events to thefirst software application, each intensity event corresponding to anintensity applied by the touch input at a corresponding time.

In some embodiments, the device receives (722) one or more instructionsfrom the first software application to generate a tactile output (e.g.,tactile output trigger 494, FIG. 3G). In response to receiving the oneor more instructions from the software application, the device generatesthe tactile output in accordance with the one or more instructions fromthe software application.

In some embodiments, while continuing to detect the first touch input onthe touch-sensitive surface, the device receives (724) one or moreinstructions to use an intensity model that corresponds to a secondintensity model identifier that is distinct from the first intensitymodel identifier (e.g., from the first software application). Forexample, in FIG. 3G, application 136 sends intensity model identifier490 to be used by contact intensity module 175 for a specified range oflocations (e.g., tracking region 492) on the display. Subsequent toreceiving the one or more instructions to use the intensity model thatcorresponds to the second intensity model identifier, the devicedetermines a second intensity applied by the first touch input on thetouch-sensitive surface, and processes the first touch input based onthe second intensity model identifier. For example, in FIG. 5S, icon510-1 and email icon 546-1 are associated with distinct intensitymodels, and the first touch input, once moved on the touch-sensitivesurface to a location that corresponds to email icon 546-1, is processedbased on an intensity model associated with email icon 546, such as asecond intensity model). In some embodiments or in some circumstances,the first intensity and the second intensity are identical. Even thoughthe first intensity and the second intensity are identical, the devicemay respond differently based on the intensity thresholds in theintensity models. In some embodiments or some circumstances, the secondintensity and the first intensity are distinct.

In some embodiments, the device determines that the first touch inputhas ceased to be detected on the touch-sensitive surface (e.g., acomplete lift-off of the first touch input or reduced intensity below adetection threshold), and detects a subsequent touch input. The deviceidentifies a second intensity model identifier based on a location ofthe subsequent touch input (e.g., a location on touch-sensitive surface451 or a corresponding location on display 450).

In some embodiments, while continuing to detect the first touch input onthe touch-sensitive surface, subsequent to processing the first touchinput based on the second intensity model identifier, the devicereceives (726, FIG. 7C) one or more instructions to use an intensitymodel that corresponds to a third intensity model identifier that isdistinct from the first intensity model identifier and the secondintensity model identifier (e.g., from the first software application).Subsequent to receiving the one or more instructions to use theintensity model that corresponds to the third intensity modelidentifier, the device determines a third intensity applied by the firsttouch input on the touch-sensitive surface; and processes the firsttouch input based on the third intensity model identifier. In someembodiments, the third intensity corresponds to the second intensity. Insome embodiments, the third intensity and the second intensity areidentical.

In some embodiments, the device determines (728) that the first touchinput has ceased to be detected on the touch-sensitive surface.Subsequent to determining that the first touch input has ceased to bedetected on the touch-sensitive surface, the device detects a secondtouch input on the touch-sensitive surface that is separate from thefirst touch input, and processes the second touch input based on thefirst intensity model identifier. For example, as shown in FIG. 5L-5N,the second intensity model expires (e.g., the device switches back tothe first intensity model) upon the lift-off of the first touch input.In some embodiments, the device selects the first intensity modelidentifier based on a location of the second touch input.

In some embodiments, processing the touch input based on the secondintensity model identifier includes (730), in accordance with the secondintensity applied by the touch input on the touch-sensitive surface andone or more thresholds associated with the second intensity modelidentifier, determining a second touch characterization parameter (e.g.,intensity stage, stage progress value, and/or transition progress valuein FIG. 5J). The second touch characterization parameter is distinctfrom the first touch characterization parameter. For example, the touchcharacterization parameter (e.g., stage, transition progress, stageprogress, etc.) has changed because the intensity model identifier (andthe corresponding intensity model) has changed regardless of whether theintensity of the touch input has changed. Processing the touch inputalso includes, subsequent to determining the second touchcharacterization parameter, sending second touch information to thefirst software application. The second touch information includes thesecond intensity model identifier and the second touch characterizationparameter.

In some embodiments, processing the touch input based on the secondintensity model identifier includes (732) foregoing generation of atactile output in accordance with the second intensity failing tosatisfying the one or more thresholds associated with the secondintensity model identifier. The electronic device (e.g., thetouch-sensitive surface of the electronic device) is configured togenerate a tactile output in accordance with the second intensitysatisfying at least one of the one or more thresholds associated withthe first intensity model identifier. For example, as shown in FIG. 5L,when intensity of contact 504 decreases below a previous releaseintensity threshold IT_(1R) but remains above a current releaseintensity threshold IT_(1R-2), the device foregoes generating a tactileoutput even though the device would have generated a tactile output ifthe previous release intensity threshold IT_(1R) were used.

In some embodiments, subsequent to detecting the first touch input, thedevice receives (734, FIG. 7B) one or more instructions to use anintensity model that corresponds to a second intensity model identifierthat is distinct from the first intensity model identifier (e.g., fromthe first software application). Subsequent to receiving the one or moreinstructions to use the intensity model that corresponds to the secondintensity model identifier, the device detects a second touch input onthe touch-sensitive surface; and, in response to detecting the secondtouch input on the touch-sensitive surface, determines a secondintensity applied by the second touch input on the touch-sensitivesurface. The device also processes the second touch input based on thesecond intensity model identifier.

It should be understood that the particular order in which theoperations in FIGS. 7A-7C have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 600, 650, 750, 800, 900, 1000, 1100, and 1200) are alsoapplicable in an analogous manner to method 700 described above withrespect to FIGS. 7A-7C. For example, the touch characterizationparameter described above with reference to method 700 optionally haveone or more of the characteristics of the intensity stage describedherein with reference to other methods described herein (e.g., method600, 650, 750, 800, 900, 1000, 1100, and 1200). For brevity, thesedetails are not repeated here.

In addition, one of ordinary skill in the art would recognize that manymodifications and variations are possible in view of the aboveteachings.

For example, in accordance with some embodiments, a method is performedat an electronic device with a touch-sensitive surface. The electronicdevice includes one or more sensors to detect intensity of contacts withthe touch-sensitive surface (e.g., the touch-sensitive surface includesone or more sensors to detect the intensity of contacts with thetouch-sensitive surface) and the electronic device stores a firstsoftware application. In some embodiments, the touch-sensitive surfaceis integrated with a display. In some embodiments, the touch-sensitivesurface is separate from the display. The method is performed usinginstructions other than instructions in the first software application(e.g., using a contact intensity module). The method includes detectinga touch input on a touch-sensitive surface; and, in response todetecting the touch input on the touch-sensitive surface, determining anintensity applied by the touch input on the touch-sensitive surface. Themethod also includes selecting a set of thresholds, from a plurality ofsets of thresholds, for the touch input. At least one set of thresholdsof the plurality of sets of thresholds includes multiple thresholds. Themethod further includes determining a touch characterization parameterof the touch input in accordance with the selected set of thresholds;and sending touch information to the first software application. Thetouch information includes the touch characterization parameter.

In some embodiments, the set of thresholds is selected based on abehavior identifier.

In some embodiments, the set of thresholds is selected based on alocation of the touch input. In some embodiments, the behavioridentifier is selected based on the location of the touch input.

In accordance with some embodiments, a method is performed at anelectronic device with a touch-sensitive surface. The electronic deviceincludes one or more sensors to detect intensity of contacts with thetouch-sensitive surface (e.g., the touch-sensitive surface includes oneor more sensors to detect the intensity of contacts with thetouch-sensitive surface) and the electronic device stores a firstsoftware application. In some embodiments, the touch-sensitive surfaceis integrated with a display. In some embodiments, the touch-sensitivesurface is separate from the display. The method is performed usinginstructions other than instructions in the first software application(e.g., using a contact intensity module). The method includes detectinga touch input on a touch-sensitive surface; and, in response todetecting the touch input on the touch-sensitive surface, determining anintensity applied by the touch input on the touch-sensitive surface. Themethod also includes selecting a set of intensity ranges, from aplurality of sets of intensity ranges, for the touch input. In someembodiments, at least one set of intensity ranges of the plurality ofsets of intensity ranges includes multiple intensity ranges. The methodfurther includes determining a touch characterization parameter of thetouch input in accordance with the selected set of intensity ranges; andsending touch information to the first software application. The touchinformation includes the touch characterization parameter.

FIG. 7D illustrate a flow diagram of a method 750 of updating a userinterface based on a touch characterization parameter in accordance withsome embodiments. The method 750 is performed at an electronic device(e.g., device 300, FIG. 3A or 3B, or portable multifunction device 100,FIG. 1A) with a display, a touch-sensitive surface, and one or moresensors to detect intensity of contacts with the touch-sensitivesurface. In some embodiments, the display is a touch-screen display andthe touch-sensitive surface is on or integrated with the display. Insome embodiments, the display is separate from the touch-sensitivesurface. Some operations in method 750 are, optionally, combined and/orthe order of some operations is, optionally, changed.

The device displays (752) a user interface of the first softwareapplication.

The device registers (754) a first intensity model identifier of aplurality of predefined force model identifiers.

In some embodiments, the device registers (756) a second intensity modelidentifier of the plurality of predefined force model identifiers. Thefirst intensity model identifier is registered with respect to a firstregion of the user interface of the first software application and thesecond intensity model identifier is registered with respect to a secondregion, of the user interface of the first software application, that isdistinct from the first region of the user interface of the firstsoftware application. In some embodiments, the first and second regionscan be overlapping, but still distinct.

In some embodiments, the device concurrently displays (758) a userinterface of a second software application while displaying the userinterface of the first software application, and registers a thirdintensity model identifier of the plurality of predefined force modelidentifiers. The third intensity model identifier is registered withrespect to a region, of the user interface of the second softwareapplication, that is distinct from the first region of the userinterface of the first software application.

Subsequent to the registering, the device receives (760) touchinformation that includes the first intensity model identifier and atouch characterization parameter; and, in response to receiving thetouch information, the device updates (762) a user interface of thefirst software application in accordance with the touch characterizationparameter.

It should be understood that the particular order in which theoperations in FIG. 7D have been described is merely exemplary and is notintended to indicate that the described order is the only order in whichthe operations could be performed. One of ordinary skill in the artwould recognize various ways to reorder the operations described herein.Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 600, 650, 700, 800, 900, 1000, 1100, and 1200) are alsoapplicable in an analogous manner to method 750 described above withrespect to FIG. 7D. For example, the touch characterization parameterdescribed above with reference to method 750 optionally have one or moreof the characteristics of the touch characterization parameter describedherein with reference to other methods described herein (e.g., methods600, 650, 700, 800, 900, 1000, 1100, and 1200). For brevity, thesedetails are not repeated here.

In addition, one of ordinary skill in the art would recognize that manymodifications and variations are possible in view of the aboveteachings.

For example, in accordance with some embodiments, a method is performedat an electronic device with a touch-sensitive surface. The electronicdevice includes one or more sensors to detect intensity of contacts withthe touch-sensitive surface (e.g., the touch-sensitive surface includesone or more sensors to detect the intensity of contacts with thetouch-sensitive surface) and the electronic device stores a firstsoftware application. In some embodiments, the touch-sensitive surfaceis integrated with a display. In some embodiments, the touch-sensitivesurface is separate from the display. The method includes registeringfirst and second intensity model identifiers of a plurality ofpredefined intensity model identifiers; and, subsequent to theregistering, receiving touch information that includes a respectiveintensity model identifier and a touch characterization parameter. Therespective intensity model identifier is the first intensity modelidentifier or the second intensity model identifier. The method alsoincludes updating a user interface of the first software application inaccordance with the touch characterization parameter and the respectiveforce model identifier.

FIGS. 8A-8C illustrate a flow diagram of a method 800 of processingtouch inputs in different regions based on distinct intensity models inaccordance with some embodiments. The method 800 is performed at anelectronic device (e.g., device 300, FIG. 3A or 3B, or portablemultifunction device 100, FIG. 1A) with a touch-sensitive surface andone or more sensors to detect intensity of contacts with thetouch-sensitive surface. In some embodiments, the device stores a firstsoftware application. In some embodiments, the device includes adisplay. In some embodiments, the display is a touch-screen display andthe touch-sensitive surface is on or integrated with the display. Insome embodiments, the display is separate from the touch-sensitivesurface. Some operations in method 800 are, optionally, combined and/orthe order of some operations is, optionally, changed.

As described below, the method 800 simplifies processing of touchinputs. The method reduces the computational burden on a softwareapplication, thereby creating a more efficient electronic device. Inaddition, the size of the software application is reduced, therebyoccupying less storage space and memory.

In some embodiments, prior to detecting a first touch input, the devicedisplays (802) a user interface that includes a first display region anda second display region (e.g., a user interface of a first softwareapplication and a user interface of a second software application, asshown in FIG. 5T). The first display region corresponds to a first touchregion of the touch-sensitive surface and the second display regioncorresponds to a second touch region of the touch-sensitive surface(e.g., in FIG. 5T, the user interface of the first software applicationcorresponds to location 502-A on touch-sensitive surface 451 and theuser interface of the second software application corresponds tolocation 502-B on touch-sensitive surface 451).

In some embodiments, the first display region corresponds (804) to auser interface of the first software application and the second displayregion corresponds to a user interface of a second software application(e.g., as shown in FIG. 5T). In some embodiments, both the first displayregion and the second display region correspond to the first softwareapplication (e.g., icon 510-1 and email icon 546-7 in FIG. 5S).

The device detects (806) a first touch input on a first touch region ofthe touch-sensitive surface (e.g., contact 522 in FIG. 5V).

The device identifies (808) a first intensity model identifier,associated with the first touch region of the touch-sensitive surface,from a plurality of predefined intensity model identifiers (e.g., anintensity model identifier that corresponds an intensity model withintensity thresholds IT₁, IT₂, and IT₃, etc. as shown in FIG. 5W).

In response to detecting the first touch input on the first touch regionof the touch-sensitive surface, the device determines (810) a firstintensity applied by the first touch input on the first touch region ofthe touch-sensitive surface (e.g., intensity of contact 522 in FIG. 5W).

In accordance with the first intensity applied by the first touch inputon the touch-sensitive surface and one or more thresholds associatedwith the first intensity model identifier, the device determines (812) afirst touch characterization parameter (e.g., stage, transition progressor stage progress).

Subsequent to determining the first touch characterization parameter,the device sends (814) first touch information to the first softwareapplication (e.g., touch information 472 is sent from contact motionmodule 130 in FIG. 3G). The first touch information includes the firsttouch characterization parameter.

The device detects (816, FIG. 8B) a second touch input on a second touchregion of the touch-sensitive surface (e.g., contact 524 in FIG. 5Y).The second touch region of the touch-sensitive surface is distinct fromthe first touch region of the touch-sensitive surface.

The device identifies (818) a second intensity model identifier,associated with the second touch region of the touch-sensitive surface,from the plurality of predefined intensity model identifiers (e.g., anintensity model identifier that corresponds an intensity model withintensity thresholds IT₁ and IT_(1R), as shown in FIG. 5Y).

In response to detecting the second touch input on the second touchregion of the touch-sensitive surface, the device determines (820) asecond intensity applied by the second touch input on the second touchregion of the touch-sensitive surface (e.g., intensity of contact 524 inFIG. 5Z).

In accordance with the second intensity applied by the second touchinput on the touch-sensitive surface and one or more thresholdsassociated with the second intensity model identifier, the devicedetermines (822) a second touch characterization parameter (e.g., stage,transition progress or stage progress).

Subsequent to determining the second touch characterization parameter,the device sends (824) second touch information to the first softwareapplication (e.g., touch information 472 is sent from contact motionmodule 130 in FIG. 3G). The second touch information includes the secondtouch characterization parameter.

In some embodiments, in response to detecting the first touch input onthe first touch region of the touch-sensitive surface, the devicegenerates (826) a first tactile output in accordance with the firstintensity applied by the first touch input on the touch-sensitivesurface and the one or more thresholds associated with the firstintensity model identifier (e.g., a tactile output is generated whenintensity of contact 522 crosses the intensity threshold IT₁ in FIG.5W). In response to detecting the second touch input on the second touchregion of the touch-sensitive surface, the device generates a secondtactile output in accordance with the second intensity applied by thesecond touch input on the touch-sensitive surface and the one or morethresholds associated with the second intensity model identifier (e.g.,a tactile output is generated when intensity of contact 524 crosses theintensity threshold IT₁ in FIG. 5Z).

In some embodiments, at least a portion of the first touch regionoverlaps (828, FIG. 8C) with at least a portion of the second touchregion. For example, as shown in FIG. 5AA, the user interface of thefirst software application and the user interface of the second softwareapplication at least partially overlap. Thus, the corresponding touchregions also overlap at least partially.

In some embodiments, the device detects (830) a third touch input on anoverlapping touch region, of the touch-sensitive surface, thatcorresponds to an overlap of the first touch region and the second touchregion (e.g., contact 526 in FIG. 5BB). The device selects an intensitymodel identifier between the first intensity model identifier and thesecond intensity model identifier for the overlapping touch region; and,in response to detecting the third touch input on the overlapping touchregion, determines a third intensity applied by the third touch input onthe overlapping touch region of the touch-sensitive surface. Inaccordance with the third intensity applied by the third touch input onthe overlapping touch region of the touch-sensitive surface and one ormore thresholds associated with the selected intensity model identifier,the device determines a third touch characterization parameter (e.g.,stage, transition progress or stage progress). Subsequent to determiningthe third touch characterization parameter, the device sends third touchinformation to the first software application. The third touchinformation includes the third touch characterization parameter. Forexample, if a first intensity model (associated with the first softwareapplication) is selected, intensity of contact 526 is processed based onthe first intensity model (e.g., FIG. 5BB). If a second intensity model(associated with the second software application) is selected, intensityof contact 526 is processed based on the second intensity model (e.g.,FIG. 5CC).

In some embodiments, both the first intensity model identifier and thesecond intensity model identifier are associated (832) with prioritiesapplicable to the overlapping touch region, and the intensity modelidentifier is selected based on the priority of the first intensitymodel identifier and the priority of the second intensity modelidentifier for the overlapping touch region (e.g., priority 489 of eachrespective intensity model 456-1, 456-2, and 456-3 in FIG. 3G). In someembodiments, the priority of an intensity model is explicitly assigned(e.g., the priority is stored in a dedicated data field, such aspriority 489 in FIG. 3G). In some embodiments, the priority of anintensity model is indicated by its position within a list or group ofintensity models (e.g., an intensity model positioned first in intensitymodels 454 in FIG. 3G has a higher priority than an intensity modelpositioned second in intensity models 454).

In some embodiments, the device foregoes (834) determination of a touchcharacterization parameter in accordance with the third intensity andone or more thresholds associated with an intensity model identifierthat has not been selected between the first intensity model identifierand the second intensity model identifier. For example, in FIG. 5CC, atouch characterization parameter is determined in accordance with thefirst intensity model, and the device foregoes determination of a touchcharacterization parameter in accordance with the second intensitymodel.

In some embodiments, the first intensity model identifier has beenselected (836) for the overlapping region. Subsequent to detecting thethird touch input, the device detects a fourth touch input on theoverlapping region of the touch-sensitive surface (e.g., contact 528,FIG. 5DD). The device selects the second intensity model identifier forthe overlapping touch region (e.g., based on one or more instructionsfrom a software application). In response to detecting the fourth touchinput on the overlapping touch region, the device determines a fourthintensity applied by the fourth touch input on the overlapping touchregion of the touch-sensitive surface. In accordance with the fourthintensity applied by the fourth touch input on the overlapping touchregion of the touch-sensitive surface and the one or more thresholdsassociated with the second intensity model identifier, the devicedetermines a fourth touch characterization parameter (e.g., an intensitystage, a stage progression value, and/or a transition progress value ofcontact 528 in FIG. 5EE). Subsequent to determining the touchcharacterization parameter, the device sends fourth touch information tothe first software application. The fourth touch information includesthe fourth touch characterization parameter.

It should be understood that the particular order in which theoperations in FIGS. 8A-8D have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 600, 650, 700, 750, 900, 1000, 1100, and 1200) are alsoapplicable in an analogous manner to method 800 described above withrespect to FIGS. 8A-8D. For example, the touch characterizationparameter described above with reference to method 800 optionally haveone or more of the characteristics of the touch characterizationparameter described herein with reference to other methods describedherein (e.g., methods 600, 650, 700, 750, 900, 1000, 1100, and 1200).For brevity, these details are not repeated here.

In addition, one of ordinary skill in the art would recognize that manymodifications and variations are possible in view of the aboveteachings.

For example, in some embodiments, a method is performed at an electronicdevice with a touch-sensitive surface. The electronic device includesone or more sensors to detect intensity of contacts with thetouch-sensitive surface (e.g., the touch-sensitive surface includes oneor more sensors to detect the intensity of contacts with thetouch-sensitive surface) and the electronic device stores a firstsoftware application. In some embodiments, the touch-sensitive surfaceis integrated with a display. In some embodiments, the touch-sensitivesurface is separate from the display. The method includes displaying auser interface that includes a first display region and a second displayregion that is distinct from the first display region; and, whiledisplaying the user interface, detecting a first touch input at alocation on the touch-sensitive surface that corresponds to the firstdisplay region (e.g., while a cursor is displayed over the first displayregion in the user interface). The method also includes, in response todetecting the first touch input at a location on the touch-sensitivesurface that corresponds to the first display region, determining afirst intensity applied by the first touch input on the touch-sensitivesurface; in accordance with the first intensity applied by the firsttouch input on the touch-sensitive surface and a first set of one ormore thresholds associated with the first display region, determining afirst touch characterization parameter (e.g., stage, transition progressor stage progress); and, subsequent to determining the first touchcharacterization parameter, sending first touch information to the firstsoftware application. The first touch information includes the firsttouch characterization parameter. The method further includes detectinga second touch input at a location on the touch-sensitive surface thatcorresponds to the second display region; and, in response to detectingthe second touch input at a location on the touch-sensitive surface thatcorresponds to the second display region, determining a second intensityapplied by the second touch input on the touch-sensitive surface; inaccordance with the second intensity applied by the second touch inputon the touch-sensitive surface and a second set of one or morethresholds associated with the second display region, determining asecond touch characterization parameter (e.g., stage, transitionprogress or stage progress); and, subsequent to determining the secondtouch characterization parameter, sending second touch information tothe first software application. The second touch information includesthe second touch characterization parameter.

FIGS. 9A-9D illustrate a flow diagram of a method 900 of processing atouch input based on a location-based intensity model in accordance withsome embodiments. The method 900 is performed at an electronic device(e.g., device 300, FIG. 3A or 3B, or portable multifunction device 100,FIG. 1A) with a touch-sensitive surface and one or more sensors todetect intensity of contacts with the touch-sensitive surface. In someembodiments, the device includes a display. In some embodiments, thedisplay is a touch-screen display and the touch-sensitive surface is onor integrated with the display. In some embodiments, the display isseparate from the touch-sensitive surface. Some operations in method 900are, optionally, combined and/or the order of some operations is,optionally, changed.

As described below, the method 900 simplifies processing of touchinputs. The method reduces the computational burden on a softwareapplication, thereby creating a more efficient electronic device. Inaddition, the size of the software application is reduced, therebyoccupying less storage space and memory.

The device detects (902) a touch input on the touch-sensitive surface(e.g., contact 536 in FIG. 5FF or contact 538 in FIG. 5HH).

In response to detecting the touch input, the device, in accordance witha determination that the touch input is at a location on thetouch-sensitive surface that is associated with a first intensity modelof a plurality of different intensity models, processes (904) the touchinput in accordance with an intensity applied by the touch input on thetouch-sensitive surface and the first intensity model (e.g., withoutprocessing the touch input in accordance with the second intensitymodel). The device, in accordance with a determination that the touchinput is at a location on the touch-sensitive surface that is associatedwith a second intensity model different from the first intensity model,processes (906) the touch input in accordance with an intensity appliedby the touch input on the touch-sensitive surface and the secondintensity model (e.g., without processing the touch input in accordancewith the first intensity model). For example, in FIG. 5FF, in accordancewith a determination that contact 536 is detected in region 542, contact536 is processed based on a first intensity model (without processingcontact 536 based on a second intensity model associated with region534-8). In FIG. 5HH, in accordance with a determination that contact 538is detected in region 534-8, contact 538 is processed based on thesecond intensity model (without processing contact 538 based on thefirst intensity model associated with region 542).

In some embodiments, the device processes (908) the touch input inaccordance with an intensity applied by the touch input and the firstintensity model includes providing contact intensity information to auser-interface application (e.g., a first software application) withoutgenerating a tactile output for the touch input (e.g., a stage progressvalue of contact 536 in FIG. 5FF is provided to application 136 in FIG.3G for determining a width of a graphical element, such as a pen stroke,and a tactile output is suppressed for contacts in region 542).Processing the touch input in accordance with an intensity applied bythe touch input and the second intensity model includes conditionallygenerating a tactile output for the touch input (e.g., in FIG. 5HH, atactile output is generated when the intensity crosses a certainintensity threshold for region 534-8).

In some embodiments, during the touch input, the electronic devicedetects (910, FIG. 9B) application of a respective intensity on thetouch-sensitive surface that is attributed to the touch input.Processing the touch input in accordance with an intensity applied bythe touch input on the touch-sensitive surface and the first intensitymodel includes providing, to a user interface application, first touchinput state information that indicates that the touch input has appliedthe respective intensity on the touch-sensitive surface withoutgenerating a tactile output for the touch input (e.g., a stage progressvalue of contact 536 in FIG. 5FF is provided to application 136, asshown in FIG. 3G, for determining a width of a graphical element, suchas a pen stroke, and a tactile output is suppressed for contacts inregion 542). Processing the touch input in accordance with an intensityapplied by the touch input on the touch-sensitive surface and the secondintensity model includes providing, to a user interface application,second touch input state information that indicates that the touch inputhas applied the respective intensity on the touch-sensitive surface andgenerating a tactile output for the touch input (e.g., a stage progressvalue, a transition progress value of contact 538 in FIG. 5HH, or adiscrete indication that a corresponding user interface element has beenactivated).

In some embodiments, the first touch input state information includes(912) a continuously variable representation of intensity of the touchinput (e.g., a value that varies as the intensity of the touch inputchanges, such as a stage progress value or a transition progress value).The second touch input state information includes an indication that theintensity of the touch input is within a range of values that correspondto activation of a user interface element that corresponds to thelocation on the touch-sensitive surface that is associated with thesecond intensity model (e.g., a discrete indication that does not varyas the intensity of the touch input changes so long as the intensity ofthe touch input is maintained within a predefined range of intensityvalues, and indicates whether a button or other affordance has beenselected).

In some embodiments, the device detects (914) a first touch input on afirst touch region (e.g., a non-tactile feedback region) of thetouch-sensitive surface (e.g., contact 536 in FIG. 5FF). The device alsoidentifies a first intensity model identifier, associated with the firsttouch region of the touch-sensitive surface, from a plurality ofpredefined intensity model identifiers (e.g., the first intensity modelidentifier corresponds to the first intensity model). In response todetecting the first touch input on the first touch region of thetouch-sensitive surface, the device determines a first intensity appliedby the first touch input on the first touch region of thetouch-sensitive surface; and, in accordance with the first intensityapplied by the first touch input on the touch-sensitive surface and thefirst intensity model identifier, processes the first touch input andforegoes generation of a tactile output for the first touch input (e.g.,a graphical element corresponding to a movement of contact 536 isdisplayed in FIG. 5GG). The device detects a second touch input on asecond touch region (e.g., a tactile feedback region) of thetouch-sensitive surface (e.g., contact 538 in FIG. 5HH). The secondtouch region of the touch-sensitive surface is distinct from the firsttouch region of the touch-sensitive surface. The device identifies asecond intensity model identifier, associated with the second touchregion of the touch-sensitive surface, from the plurality of predefinedintensity model identifiers (e.g., the second intensity model identifiercorresponds to the second intensity model). In response to detecting thesecond touch input on the second touch region of the touch-sensitivesurface, the device determines a second intensity applied by the secondtouch input on the second touch region of the touch-sensitive surface;and, in accordance with the second intensity applied by the second touchinput on the touch-sensitive surface and the second intensity modelidentifier, processes the second touch input (e.g., a correspondingcharacter is inserted into a document, as shown in FIG. 5HH) andconditionally generates a tactile output for the second touch input inaccordance with the second touch input and one or more model parametersassociated with the second intensity model identifier (e.g., one or moremodel parameters in the second intensity model).

In some embodiments, conditionally generating the tactile output for thesecond touch input includes (916, FIG. 9C) foregoing generation of thetactile output for the second touch input in accordance with adetermination that the second touch input fails to satisfy one or morecriteria corresponding to the one or more model parameters associatedwith the second intensity model identifier. For example, in someembodiments, a tactile output is not generated when a touch input doesnot cross an intensity threshold.

In some embodiments, the first intensity model identifier indicates(918) that no tactile output is to be generated for a touch input on thefirst touch region of the touch-sensitive surface.

In some embodiments, the electronic device stores (920) a first softwareapplication, and the first user input and the second user input aredetected while the electronic device displays a user interface of thefirst software application (e.g., a user interface of a word processingsoftware application as shown in FIGS. 5FF-5JJ).

In some embodiments, the user interface of the first softwareapplication includes (922) a handwriting input tool region (e.g., acharacter input tool region) (e.g., a handwriting input tool region 540in FIG. 5FF). In some embodiments, the handwriting input tool region isdistinct from the user interface of the first software application andthe handwriting input tool region overlays the user interface of thefirst software application (e.g., a user interface of a word processingsoftware application). In some embodiments, the handwriting input toolregion is separate from the user interface of the first softwareapplication.

In some embodiments, the handwriting input tool region includes (924) aplurality of selection regions, and at least a respective selectionregion of the one or more selection regions corresponds to the secondtouch region (e.g., a tactile feedback region) of the touch-sensitivesurface. For example, in FIG. 5JJ, selection region 532-1 corresponds totouch region 534-1. In some embodiments, the one or more selectionregions collectively correspond to the second touch region.

In some embodiments, the handwriting input tool region includes (926) ahandwriting input region, and the handwriting input region correspondsto the first touch region (e.g., a potential non-tactile feedbackregion) of the touch-sensitive surface. For example, in FIG. 5FF,handwriting input region 530 corresponds to region 542.

In some embodiments, in response to detecting the first touch input onthe first touch region of the touch-sensitive surface, the devicedisplays (928) one or more graphical elements (e.g., pen strokes, suchas 544 in FIG. 5GG) in the handwriting input region in accordance withthe first touch input.

In some embodiments, in response to detecting the first touch input onthe first touch region (e.g., a potential non-tactile feedback region}of the touch-sensitive surface, the device displays (930, FIG. 9D) aplurality of groups of characters in the plurality of selection regions(e.g., characters in selection regions 532-5 through 532-8 in FIG. 5GG).A respective group of characters is selected based on the one or moredisplayed graphical elements in the handwriting input region, and therespective group of characters is displayed in a respective selectionregion (e.g., characters, such as Chinese, Korean, and Japanesecharacters, that correspond to displayed pen strokes). In response todetecting the second user input on the second touch region of thetouch-sensitive surface at a location that corresponds to the respectiveselection region, the device selects the respective group of characters(e.g., in FIG. 5HH, in response to contact 538 in region 534-8, thecharacter in a corresponding selection region 532-8 is selected). Insome embodiments, in response to detecting the second user input on thesecond touch region of the touch-sensitive surface at a location thatcorresponds to the respective selection region and subsequent toselecting the respective group of characters, the device displays therespective group of characters in the user interface of the firstsoftware application other than the handwriting input tool region (e.g.,in FIG. 5HH, in response to contact 538 on region 534-8, the characterin a corresponding selection region 532-8 is inserted into thedocument).

In some embodiments, processing the first touch input detected on thefirst touch region of the touch-sensitive surface includes (932)determining that the first touch input has satisfied a drawing intensitythreshold; and, in accordance with a determination that the first touchinput has exceeded the drawing intensity threshold, displaying the oneor more graphical elements in the handwriting input region in accordancewith the first touch input. A first terminal region of a respectivegraphical element of the one or more graphical elements corresponds to alocation where the first touch input has been determined to satisfy thedrawing intensity threshold. For example, as shown in FIG. 5II, drawingof a graphical element is initiated in response to intensity of contact536 satisfying the drawing intensity threshold IT_(D).

In some embodiments, processing the first touch input detected on thefirst touch region of the touch-sensitive surface includes (934) whilecontinuing to detect the first touch input on the touch-sensitivesurface, continuing to update the one or more graphical elements in thehandwriting input region in accordance with the first touch inputregardless of whether the first touch input has ceased to satisfy thedrawing intensity threshold. For example, as shown in FIG. 5II, once thedrawing of a graphical element is initiated, the drawing continues evenif intensity of contact 536 falls below the drawing intensity threshold.

In some embodiments, processing the first touch input detected on thefirst touch region of the touch-sensitive surface includes (936)detecting that the first touch input has ceased to be detected on thetouch-sensitive surface and detecting that the first touch input hasresumed contact with the touch-sensitive surface within a predefinedtime interval subsequent to the first touch input ceasing to be detectedon the touch-sensitive surface. Processing the first touch input alsoincludes, in response to the first touch input resuming contact with thetouch-sensitive surface within the predefined time interval subsequentto the first touch input ceasing to be detected on the touch-sensitivesurface, updating the one or more graphical elements in the handwritinginput region in accordance with the first touch input. For example, asshown in FIG. 5JJ, even if contact 536 breaks contact withtouch-sensitive surface 451 while following a path on touch-sensitivesurface 451 and subsequent contact 544 continues the path, the paths ofcontact 536 and contact 544 are treated as a single path.

It should be understood that the particular order in which theoperations in FIGS. 9A-9D have been described is merely exemplary and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 600, 650, 700, 750, 800, 1000, 1100, and 1200) are alsoapplicable in an analogous manner to method 900 described above withrespect to FIGS. 9A-9D. For example, the intensity model described abovewith reference to method 900 optionally have one or more of thecharacteristics of the intensity model described herein with referenceto other methods described herein (e.g., methods 600, 650, 700, 750,800, 1000, 1100, and 1200). For brevity, these details are not repeatedhere.

FIGS. 10A-10D illustrate a flow diagram of a method 1000 of processing atouch input based on latching of the touch input in accordance with someembodiments. The method 1000 is performed at an electronic device (e.g.,device 300, FIG. 3A or 3B, or portable multifunction device 100, FIG.1A) with a touch-sensitive surface and one or more sensors to detectintensity of contacts with the touch-sensitive surface. The devicestores a first software application. In some embodiments, the deviceincludes a display. In some embodiments, the display is a touch-screendisplay and the touch-sensitive surface is on or integrated with thedisplay. In some embodiments, the display is separate from thetouch-sensitive surface. Some operations in method 1000 are, optionally,combined and/or the order of some operations is, optionally, changed.

As described below, the method 1000 simplifies processing of touchinputs. The method reduces the computational burden on a softwareapplication, thereby creating a more efficient electronic device. Inaddition, the size of the software application is reduced, therebyoccupying less storage space and memory.

The device displays (1002) a user interface that includes two or moredisplay regions, including a first display region and a second displayregion (e.g., icon 510-1 and email icon 546-7 in FIG. 5KK).

While displaying the user interface, the device detects (1004) a firstintensity applied by a touch input at a first location on thetouch-sensitive surface that corresponds to the first display region(e.g., intensity of contact 548 in FIG. 5LL).

In some embodiments, in response to detecting the first intensityapplied by the touch input, the device processes (1006) the firstintensity in accordance with the one or more intensity thresholdsassociated with the first display region (e.g., in FIG. 5LL, intensityof contact 548 is processed in accordance with intensity thresholdsassociated with icon 510-1).

In some embodiments, processing the first intensity in accordance withthe one or more intensity thresholds associated with the first displayregion includes (1008) displaying an animation that corresponds to achange in intensity applied by the touch input from a prior intensity tothe first intensity (e.g., from intensity, detected prior to detectingthe first intensity, to the first intensity) (e.g., icon 510-1 isvisually distinguished in FIG. 5MM). For example, the device displays apre-latch animation that indicates that the user is interacting with aparticular control in the first display region such as a change inappearance of a first button in the first display region that isproximate to a focus selector. This pre-latch animation is optionallypresented at a rate that is dependent on a rate of change in theintensity of the contact or magnitude of intensity of the contact so asto provide feedback to the user as to their progress toward activatingthe button or latching onto the button.

The device detects (1010) a movement of the touch input across thetouch-sensitive surface from the first location on the touch-sensitivesurface to a second location on the touch-sensitive surface thatcorresponds to the second display region (e.g., while a cursor isdisplayed over the first display region) (e.g., movement of contact 548in FIG. 5NN). In some embodiments, the movement of the touch inputacross the touch-sensitive surface from the first location on thetouch-sensitive surface to the second location on the touch-sensitivesurface is detected after detecting the first intensity applied by thetouch input at the first location.

In some embodiments, in response to detecting the movement of the touchinput from the first location on the touch-sensitive surface to thesecond location on the touch-sensitive surface, the device displays(1012) an animation that corresponds to a change in intensity applied bythe touch input (e.g., from an intensity, detected prior to detectingthe first intensity, to the first intensity). For example, the devicedisplays a pre-latch animation that indicates that the user isinteracting with a particular control in the second display region suchas a change in appearance of a second button in the second displayregion that is proximate to a focus selector. This pre-latch animationis optionally presented at a rate that is independent of a rate ofchange in the intensity of the contact so as to catch the animation upto a current point in the pre-activation animation that corresponds to acurrent intensity of the contact.

After detecting the movement of the touch input from the first locationon the touch-sensitive surface to the second location on thetouch-sensitive surface, the device detects (1014) a second intensityapplied by the touch input at the second location on the touch-sensitivesurface (e.g., intensity of contact 548 in FIG. 5OO).

In response to detecting the second intensity applied by the touch inputat the second location on the touch-sensitive surface, the device, inaccordance with a determination that the first intensity does notsatisfy a first intensity threshold (e.g., a latching intensitythreshold), processes (1016, FIG. 10B) the second intensity inaccordance with one or more intensity thresholds associated with thesecond display region (e.g., in FIG. 5PP, contact 548 is processed basedon intensity thresholds associated with email icon 546-7).

In some embodiments, the first intensity threshold is (1018) distinctfrom an activation intensity threshold. In some embodiments, the devicedetects a touch input at a location on the touch-sensitive surface thatcorresponds to the first display region; and, in response to detectingthe touch input, determines an intensity applied by the touch input onthe touch-sensitive surface. In accordance with a determination that theintensity applied by the touch input on the touch-sensitive surfacesatisfies the activation intensity threshold, the device initiatesactivation of the first software element (e.g., sends to the firstsoftware element one or more instructions to activate the first softwareelement).

In some embodiments, processing the second intensity in accordance withthe one or more intensity thresholds associated with the second displayregion includes (1020) displaying an animation that corresponds to achange in intensity applied by the touch input on the touch-sensitivesurface (e.g., from the first intensity to the second intensity). Forexample, the device displays a pre-activation animation corresponding tothe user interface element in the second display region such as a buttonpress animation that indicates that a button in the second displayregion will be activated if the intensity of the contact increases abovethe button-activation intensity threshold. This pre-activation animationis optionally presented at a rate that is dependent on a rate of changein the intensity of the contact or magnitude of intensity of the contactso as to provide feedback to the user as to their progress towardactivating the button.

In some embodiments, processing the second intensity in accordance withone or more intensity thresholds associated with the second displayregion includes (1022) activating a second control associated with thesecond display region (e.g., when the contact meets activation criteriathat include a criterion that is met when the contact exceeds anactivation intensity threshold for the second control).

In response to detecting the second intensity applied by the touch inputat the second location on the touch-sensitive surface, the device, inaccordance with a determination that the first intensity satisfies thefirst intensity threshold, processes (1024) the second intensity inaccordance with one or more intensity thresholds associated with thefirst display region (e.g., in FIG. 5SS, contact 548 is processed basedon intensity thresholds associated with icon 510-1).

In some embodiments, the one or more intensity thresholds associatedwith the first display region are (1026, FIG. 10C) different from theone or more intensity thresholds associated with the second displayregion (e.g., the first display region and the second display region areboth associated with different intensity models).

In some embodiments, the one or more intensity thresholds associatedwith the first display region are (1028) the same as the one or moreintensity thresholds associated with the second display region (e.g.,the first display region and the second display region are bothassociated with a same intensity model).

In some embodiments, processing the second intensity in accordance withthe one or more intensity thresholds associated with the first displayregion includes (1030) displaying an animation that corresponds to achange in intensity applied by the touch input on the touch-sensitivesurface (e.g., from the first intensity to the second intensity). Forexample, the device displays a pre-activation animation corresponding toa user interface element in the first display region such as apre-button press animation that indicates that a button in the firstdisplay region will be activated if the intensity of the contactincreases above a button-activation intensity threshold. Thispre-activation animation is optionally presented at a rate that isdependent on a rate of change in the intensity applied by the touchinput or magnitude of the intensity applied by the touch input so as toprovide feedback to the user as to their progress toward activating thebutton.

In some embodiments, processing the second intensity in accordance withone or more intensity thresholds associated with the first displayregion includes (1032) activating a first control associated with thefirst display region (e.g., when the contact meets activation criteriathat include a criterion that is met when the contact exceeds anactivation intensity threshold for the first control).

In some embodiments, the device sends (1034, FIG. 10B) first touchinformation to the first software application in accordance with adetermination that the first intensity satisfies a reporting intensitythreshold that is distinct from the first intensity threshold. The firsttouch information includes one or more touch parameters that correspondto the first intensity. In some embodiments, the first intensitythreshold requires a higher intensity than the reporting intensitythreshold. In some embodiments, the method includes, in accordance witha determination that the first intensity does not satisfy the reportingintensity threshold, foregoing sending the first touch information. Thedevice sends second touch information to the first software applicationin accordance with a determination that the second intensity satisfiesthe reporting intensity threshold. The second touch information includesone or more touch parameters that correspond to the second intensity.

In some embodiments, a first display region of the two or more displayregions is associated (1036, FIG. 10D) with a first software element ofthe first software application and a second display region of the two ormore display regions is associated with a second software element of thesecond software application.

In some embodiments, in accordance with the determination that the firstintensity does not satisfy the first intensity threshold (e.g., notlatched on the first display region), the second touch information issent (1038) to the second software element without sending the secondtouch information to the first software element.

In some embodiments, in accordance with the determination that the firstintensity satisfies the first intensity threshold (e.g., latched on thefirst display region), the second touch information is sent (1040) tothe first software element without sending the second touch informationto the second software element.

In some embodiments, in accordance with the determination that the firstintensity satisfies the reporting intensity threshold, the first touchinformation is sent (1042) to the first software element. In someembodiments, the first touch information is sent to the first softwareelement without sending the first touch information to the secondsoftware element.

In some embodiments, in response to detecting the touch input, thedevice updates (1044, FIG. 10B) the user interface in accordance withthe first touch information using the first software application priorto the first intensity satisfying the first intensity threshold. In someembodiments, updating the user interface in accordance with the firsttouch information using the first software application includesdisplaying an animation that indicates detection of the touch input. Forexample, the animation may be used to inform the user as to theintensity required to achieve latching. In some embodiments, updatingthe user interface in accordance with the first touch information usingthe first software application includes displaying an animation thatindicates increase and/or decrease in an intensity applied by the touchinput on the touch-sensitive surface.

It should be understood that the particular order in which theoperations in FIGS. 10A-10D have been described is merely exemplary andis not intended to indicate that the described order is the only orderin which the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 600, 650, 700, 750, 800, 900, 1100, and 1200) are alsoapplicable in an analogous manner to method 1000 described above withrespect to FIGS. 10A-10D. For example, the intensity thresholdsdescribed above with reference to method 1000 optionally have one ormore of the characteristics of the intensity thresholds described hereinwith reference to other methods described herein (e.g., methods 600,650, 700, 750, 800, 900, 1100, and 1200). For brevity, these details arenot repeated here.

FIGS. 11A-11C are flow diagrams illustrating a method 1100, performed byan electronic device of routing tactile outputs among a plurality ofinput devices that are in communication with the electronic device. Insome embodiments, the electronic device includes one or more of theinput devices (e.g., a touch-screen display or trackpad). As describedbelow, the method 1100 provides tactile feedback, herein called atactile output, to the same input device as the input device from whichan input was received that triggered a reaction by an applicationexecuted by the electronic device. The method enables and/or facilitatesthe use of multiple input devices at which tactile outputs can begenerated, and enables the use of tactile feedback by a broad range ofapplications that can be executed by the electronic device. The methodalso reduces the cognitive burden on a user when performing an operationin conjunction with an application, by providing tactile feedback thatfacilitates efficient use of the electronic device and the application,thereby creating a more efficient human-machine interface. Forbattery-operated electronic devices, enabling a user to perform anoperation in accordance with a selected mode of operation faster andmore efficiently conserves power and increases the time between batterycharges.

Optionally, but typically, the device displays (1102), on a display thatis part of the device or in communication with the device, a userinterface for the electronic device. Examples of such user interfacesbeing displayed by an electronic device are shown in FIGS. 5A-5S. Insome embodiments, the user interface includes two of more displayregions, including a first display region and a second display region.Typically, the user interface continues to be displayed throughoutperformance of method 1100, although content of the user interface maychange, for example in response to inputs by a user of the electronicdevice. Furthermore, the device is in communication with a plurality ofinput devices including a first input device (e.g., a first input devicehaving a first touch-sensitive surface that includes one or more sensorsto detect intensity of contacts with the first touch-sensitive surface)that is configured to generate tactile outputs in response to inputs anda second input device (e.g., a second input device having a secondtouch-sensitive surface that includes one or more sensors to detectintensity of contacts with the second touch-sensitive surface) that isconfigured to generate tactile outputs. Non-limiting examples of thefirst input device are a trackpad and touch-sensitive display, andnon-limiting examples of the second input device are a trackpad andtouch-sensitive display.

The device receives (1104) an indication of an input detected by arespective input device of the plurality of input devices. In someembodiments, the touch input can be a finger touch input by a person'sfinger, or a stylus touch input. In response to receiving the indicationof the input, the device provides (1106) information describing theinput to an application running on the device that enables theapplication to react to the input. For example, a touch event specifyingthe movement (if any), location, intensity, etc. of one or more contactsis provided to an application. In some embodiments, the touch eventprovided to the application includes a plurality of lists, such as twoor more of: a list of all touches currently detected by input devices incommunication with the device, a list of all touches associated with aparticular view of an application, and a list of changed touches,comprising touches for which at least one parameter or aspect (e.g.,location and/or intensity) has changed since a prior touch event wasprovided to the application.

The device receives (1108) a reaction to the input from the applicationthat indicates that a tactile output is to be generated in response tothe input. For example, in some circumstances, the input includes (1110)lateral movement, and the reaction of the application includes moving agraphical user interface object across a display.

In response to receiving the reaction to the input from the application,the device causes (1112) the generation of a respective tactile output.More specifically, in accordance with a determination (1114) that thereaction was triggered by the first input device, the respective tactileoutput is generated at the first input device (e.g., touch-screendisplay 340, FIG. 3B) based on the reaction to the input from theapplication; and in accordance with a determination (1116) that thereaction was triggered by the second input device (e.g., trackpad 332,FIG. 3B), the respective tactile output is generated at the second inputdevice based on the reaction to the input from the application.

In some embodiments, in response to receiving the reaction to the inputfrom the application, the device determines (1118) which input device ofthe plurality of input devices triggered the reaction from theapplication. For example, in the context of FIG. 3B, which includes botha trackpad 332 and touch-sensitive display 340, the device determineswhich of the trackpad and touch-sensitive display triggered the reactionfrom the application. In some embodiments, that determination is made bythe application. For example, in some circumstances, inputs may bereceived concurrently, or during overlapping time periods, from two ormore input devices, and the application may make the determination as towhich of the input devices triggered the reaction. Continuing with thisexample, the input from one of the input devices may be used by theapplication to select a mode of operation or select an option, while theinput from another one of the input devices triggers the reaction, andit is the application that makes this determination.

More specifically, and consistent with what has been described above, insome embodiments, when the reaction was triggered by the first inputdevice, the tactile output at the first input device is generated (1120)without causing generation of any tactile output at the second inputdevice, and when the reaction was triggered by the second input device,the tactile output at the second input device is generated (1122)without causing generation of any tactile output at the first inputdevice.

In some embodiments, the information describing the input includes(1124) a respective identifier for the respective device, the reactionto the input includes the respective identifier for the respectivedevice, and determining that the reaction was triggered by the firstinput device includes determining (1126) that the respective identifieris an identifier for the first input device. Similarly, thedetermination that the reaction was triggered by the second input deviceincludes determining (1128) that the respective identifier is anidentifier for the second input device. Furthermore, in someembodiments, the respective identifier is (1130) part of an input event(e.g., a touch event) that describes the input. Thus, in suchembodiments, the module(s) that initially process inputs from the inputdevices and that generate or cause the generation of tactile outputs,and the application that responds to such inputs, utilize distinctidentifiers for each of the input devices in communication with thedevice. In some embodiments, this facilitates routing tactile feedbackto the input device determined (e.g., see 1118) to have triggered thereaction from the application.

In some embodiments, the device is configured (1132) to cause thegeneration of tactile outputs at the plurality of input devices inresponse to changes in intensity of contacts on the input devices inaccordance with one or more intensity models. For example, this isdescribed in greater detail above with reference to methods 600, 700,800, 900 and 1000. Furthermore, in some circumstances, the respectivetactile output is generated (1134) without reference to the one or moreintensity models. The following is a non-limiting set of examples ofsuch circumstances. In one example, the tactile feedback requested by anapplication is in response to a lateral movement of a touch input thatmoves one object adjacent to, near, over or partially over anotherobject, and the tactile feedback concerns the lateral movement, but notthe intensity of the contact corresponding to the touch input. Anotherexample in a respective tactile output is generated without reference tothe one or more intensity models is a movement of the touch input thatresults in the application adjusting the position of an object, forexample “snapping” the object to a horizontal and/or vertical positiondefined by a grid or guide. Yet another example in a respective tactileoutput is generated without reference to the one or more intensitymodels is a movement of the touch input that violates a predefined ruleor condition (e.g., a movement of the touch input that is construed bythe application to be an instruction to move an object over anotherobject, in violation of a rule), or that satisfies a predefined rule orcondition (e.g., a movement of the touch input that is construed by theapplication to be an instruction to move an object over another object,thereby initiating or enabling an action, such as adding the object to acollection, changing a property of the moved object or the other object,etc.

It should be understood that the particular order in which theoperations in FIGS. 11A-11C have been described is merely exemplary andis not intended to indicate that the described order is the only orderin which the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 600, 650, 700, 750, 800, 900, 1000, and 1200) are alsoapplicable in an analogous manner to method 1100 described above withrespect to FIGS. 11A-11C. For example, the contacts, user interfaceobjects, tactile outputs, intensity thresholds, and focus selectorsdescribed above with reference to method 1100 optionally have one ormore of the characteristics of the contacts, user interface objects,tactile outputs, intensity thresholds, and focus selectors describedherein with reference to other methods described herein (e.g., methods600, 650, 700, 750, 800, 900, 1000, and 1200). For brevity, thesedetails are not repeated here.

FIGS. 12A-12C are flow diagrams illustrating a method 1200, performed byan electronic device that is in communication with one or more inputdevices that are configured to generate tactile outputs in response toinputs, in which the device conditionally cancels or forgoes generationof a tactile output corresponding to the reaction from the applicationin accordance with a determination that tactile output criteria have notbeen met. The tactile output criteria include a criterion (sometimescalled a timing criterion or latency criterion) that is met when aninput time is less than a predetermined amount of time before the outputtime. In some embodiments, the electronic device includes one or more ofthe input devices (e.g., a touch-screen display or trackpad).

Method 1200 enables and/or facilitates the use of complex applicationsoftware for which tactile outputs can be generated, and enables the useof tactile feedback by a broad range of applications that can beexecuted by the electronic device. The method also reduces the cognitiveburden on a user when performing an operation in conjunction with anapplication, by providing limiting the provision of tactile feedback tocircumstances in which the tactile feedback will be intuitivelymeaningful, and forgoing the provision of tactile feedback incircumstances in which the tactile feedback might be confusing to theuser or even mis-informative due to latency between the input that causethe application's reaction and the time at which the tactile outputwould be generated.

In a non-limiting example, the generation of a reaction to an input byan application may be delayed by connectivity issues, a stalled processexecuting on the same device or another device, a failed procedure callto another application or module, or any of a potentially large numberof circumstances. In at least some situations, when that delay resultsin a time difference between the input time, at which the input wasdetected by a respective input device, and an output time for thereaction by the application that exceeds a predetermined amount of time,the method forgoes generation of the tactile output corresponding to thereaction from the application.

Method 1200 therefore facilitates efficient use of the electronic deviceand the application, thereby creating a more efficient human-machineinterface. For battery-operated electronic devices, enabling a user toperform an operation in accordance with a selected mode of operationfaster and more efficiently conserves power and increases the timebetween battery charges.

Optionally, but typically, the device displays (1202), on a display thatis part of the device or in communication with the device, a userinterface for the electronic device. Examples of such user interfacesbeing displayed by an electronic device are shown in FIGS. 5A-5S. Insome embodiments, the user interface includes two of more displayregions, including a first display region and a second display region.Typically, the user interface continues to be displayed throughoutperformance of method 1200, although content of the user interface maychange, for example in response to inputs by a user of the electronicdevice. Furthermore, the device is in communication with a plurality ofinput devices including a first input device (e.g., a first input devicehaving a first touch-sensitive surface that includes one or more sensorsto detect intensity of contacts with the first touch-sensitive surface)that is configured to generate tactile outputs in response to inputs anda second input device (e.g., a second input device having a secondtouch-sensitive surface that includes one or more sensors to detectintensity of contacts with the second touch-sensitive surface) that isconfigured to generate tactile outputs. Non-limiting examples of aninput device of the one or more input devices is a trackpad ortouch-sensitive display.

The device receives (1204) an indication of an input detected by arespective input device of the one or more of input devices. In someembodiments, the touch input can be a finger touch input by a person'sfinger, or a stylus touch input. In response to receiving the indicationof the input, the device provides (1206) information describing theinput to an application running on the device that enables theapplication to react to the input. For example, a touch event specifyingthe movement (if any), location, intensity, etc. of one or more contactsis provided to an application. In some embodiments, the touch eventprovided to the application includes a plurality of lists, such as twoor more of: a list of all touches currently detected by input devices incommunication with the device, a list of all touches associated with aparticular view of an application, and a list of changed touches,comprising touches for which at least one parameter or aspect (e.g.,location and/or intensity) has changed since a prior touch event wasprovided to the application.

The device receives (1208) a reaction to the input from the applicationthat indicates that a tactile output is to be generated in response tothe input. For example, in some circumstances, the input includesmovement, and the reaction of the application includes moving agraphical user interface object across a display.

In response to receiving the reaction to the input from the application,the device performs (1210) a set of operations, including comparing(1212) an input time for the reaction to an output time for thereaction, wherein the input time for the reaction corresponds to a timeat which the input was detected by the respective input device, and theoutput time for the reaction corresponds to a time at which a tactileoutput corresponding to the reaction is configured to be generated atthe respective input device. Stated another way, the comparing operationcompares an input time for the input detected by the respective inputdevice to an output time for the reaction. The set of operationsperformed by the device also includes determining (1214) whether tactileoutput criteria have been met, wherein the tactile output criteriainclude a criterion that is met when an input time is less than apredetermined amount of time before the output time, causing generation(1216), at the respective input device, of a tactile outputcorresponding to the reaction from the application in accordance with adetermination that the tactile output criteria have been met, andforgoing generation (1218)), at the respective input device, of thetactile output corresponding to the reaction from the application inaccordance with a determination that the tactile output criteria havenot been met. In some embodiments, the tactile output criteria alsoinclude tactile output routing criteria to enable tactile outputs to begenerated at the device that detected an input that caused the tactileoutput to be generated (e.g., as described in greater detail above withreference to method 1100.

In some embodiments, the input time for the reaction (e.g., the inputtime for the input detected by the respective input device) is stored(1222) by the device in response to receiving the indication of theinput from the respective input device. Thus, the device monitors inputtimes to ensure that stale tactile outputs are not generated at therespective input device.

In some embodiments, the input time for the reaction (e.g., the inputtime for the input detected by the respective input device) is received(1220) from the application along with the reaction to the input. Thus,in these embodiments, the application monitors input times and providesinput time information to the device to ensure that stale tactileoutputs are not generated at the respective input device.

In some embodiments, the output time for the reaction is (1224) a timeat which a tactile output for the reaction would be scheduled by thedevice. Thus, in at least some such embodiments, the output time is acomputed time, in the near future, that is either the time at which thetactile output would be generated, or is a time that the tactile outputis scheduled to be generated. The later times are not necessarilyidentical, since in some embodiments the actual tactile outputgeneration time may not be entirely predictable or schedulable.

In some embodiments, the output time for the reaction is (1226) a timeat which the application requested that a tactile output be generated atthe respective input device. In these embodiments, the output time isdetermined by the time that the application posts, transfers, orotherwise communicates its request to generate a tactile output.

In some embodiments, the device is configured (1230) to cause thegeneration of tactile outputs at the plurality of input devices inresponse to changes in intensity of contacts on the input devices inaccordance with one or more intensity models. For example, this isdescribed in greater detail above with reference to methods 600, 700,800, 900 and 1000. Furthermore, in some circumstances, the respectivetactile output is generated (1234) without reference to the one or moreintensity models. A non-limiting set of examples of such circumstancesare described above with respect to operation 1132 of method 1100.

In some embodiments, the predetermined amount of time that is used todetermine whether the tactile output criteria have been met has a firstvalue in a normal mode of operation, and a second value, larger than thefirst value, in an accessibility mode of operation. In theseembodiments, in the accessibility mode, a slower reaction time to userinputs is acceptable, as is slower tactile output in response to userinputs at the one or more input devices.

It should be understood that the particular order in which theoperations in FIGS. 12A-12C have been described is merely exemplary andis not intended to indicate that the described order is the only orderin which the operations could be performed. One of ordinary skill in theart would recognize various ways to reorder the operations describedherein. Additionally, it should be noted that details of other processesdescribed herein with respect to other methods described herein (e.g.,methods 600, 650, 700, 750, 800, 900, 1000, and 1100) are alsoapplicable in an analogous manner to method 1200 described above withrespect to FIGS. 12A-12C. For example, the contacts, user interfaceobjects, tactile outputs, intensity thresholds, and focus selectorsdescribed above with reference to method 1200 optionally have one ormore of the characteristics of the contacts, user interface objects,tactile outputs, intensity thresholds, and focus selectors describedherein with reference to other methods described herein (e.g., methods600, 650, 700, 750, 800, 900, 1000, and 1100). For brevity, thesedetails are not repeated here.

The operations in the information processing methods described aboveare, optionally implemented by running one or more functional modules ininformation processing apparatus such as general purpose processors(e.g., as described above with respect to FIGS. 1A, 3A, and 3B) orapplication specific chips.

In accordance with some embodiments, FIG. 13 shows a functional blockdiagram of electronic device 1300 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software,firmware, or a combination thereof to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 13 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 13, electronic device 1300 includes touch-sensitivesurface unit 1304 configured to receive contacts and one or more sensorunits 1306 configured to detect intensity of contacts with thetouch-sensitive surface unit 1304. Processing unit 1308 is coupled withthe touch-sensitive surface unit 1304 and the one or more sensor units1306. In some embodiments, electronic device 1300 includes display unit1302 configured to display a user interface, and processing unit 1308 iscoupled with display unit 1302. In some embodiments, processing unit1310 includes: detecting unit 1310, determining unit 1312, identifyingunit 1314, touch input processing unit 1316, sending unit 1318, andrepeating unit 1320.

The processing unit 1308 configured to: detect a touch input on thetouch-sensitive surface unit 1304 (e.g., with the detecting unit 1310and/or the touch-sensitive surface unit 1304); in response to detectingthe touch input on the touch-sensitive surface unit 1304, determine anintensity of the touch input on the touch-sensitive surface unit 1304(e.g., with the determining unit 1312, the touch-sensitive surface unit1304, and/or the sensor units 1306); and, in accordance with theintensity of the touch input on the touch-sensitive surface unit 1304and one or more preselected intensity thresholds, determine an intensitystage of the touch input (e.g., with the determining unit 1312), whereinthe intensity stage of the touch input is selected from a plurality ofpredefined intensity stages; and process the touch input based on theintensity stage of the touch input (e.g., with the touch inputprocessing unit 1316).

In some embodiments, the electronic device 1300 stores a first softwareapplication, and the intensity stage of the touch input is determined bya contact intensity module that is distinct and separate from the firstsoftware application.

In some embodiments, the processing unit 1308 is configured to send tothe first software application from the contact intensity module touchinformation that identifies the intensity stage of the touch input(e.g., with the sending unit 1318).

In some embodiments, the processing unit 1308 is configured to sendinformation from the contact intensity module, the informationindicating that the intensity of the touch input is available to thefirst software application (e.g., with the sending unit 1318).

In some embodiments, the processing unit 1308 is configured to repeatthe operations of determining an intensity of the touch input,determining an intensity stage of the touch input, and sending touchinformation while the touch input is detected on the touch-sensitivesurface unit 1304 (e.g., with the repeating unit 1320, the determiningunit 1312, the sending unit 1318, and/or the touch-sensitive surfaceunit 1304).

In some embodiments, the processing unit 1308 is configured to determineone or more intensity-based progress values of the touch input based onan intensity range associated with the determined intensity stage (e.g.,with the determining unit 1312); and send touch information to the firstsoftware application (e.g., with the sending unit 1318), wherein thetouch information includes the one or more intensity-based progressvalues of the touch input and information identifying the intensitystage of the touch input.

In some embodiments, the one or more intensity-based progress values ofthe touch input include a transition progress value of the touch input.

In some embodiments, the one or more intensity-based progress values ofthe touch input include a stage progress value of the touch input.

In some embodiments, the processing unit 1308 is configured to, inresponse to detecting the touch input, determine a first intensityapplied by the touch input on the touch-sensitive surface unit 1304(e.g., with the determining unit 1312); in accordance with adetermination that the first intensity applied by the touch input on thetouch-sensitive surface unit 1304 does not satisfy a stage activationintensity threshold for a second intensity stage, determine that thetouch input is in a first intensity stage that is distinct from thesecond intensity stage (e.g., with the determining unit 1312);subsequent to determining that the touch input is in the first intensitystage, determine a second intensity applied by the touch input on thetouch-sensitive surface unit 1304 (e.g., with the determining unit1312), wherein the second intensity is distinct from the firstintensity; and, in accordance with a determination that the secondintensity applied by the touch input on the touch-sensitive surface unit1304 satisfies the stage activation intensity threshold for the secondintensity stage, determine that the touch input is in the secondintensity stage (e.g., with the determining unit 1312).

In some embodiments, the processing unit 1308 is configured to,subsequent to determining that the touch input is in the secondintensity stage, determine a third intensity applied by the touch inputon the touch-sensitive surface unit 1304 (e.g., with the determiningunit 1312), wherein the third intensity is distinct from the secondintensity; in accordance with a determination that the third intensitydoes not satisfy a stage release intensity threshold for the secondintensity stage, distinct from the stage activation intensity thresholdfor the second intensity stage, determine that the touch input remainsin the second intensity stage (e.g., with the determining unit 1312);subsequent to determining that the touch input remains in the secondintensity stage, determine a fourth intensity applied by the touch inputon the touch-sensitive surface unit 1304 (e.g., with the determiningunit 1312), wherein the fourth intensity is distinct from the thirdintensity; and, in accordance with a determination that the fourthintensity satisfies the stage release intensity threshold for the secondintensity stage, determine that the touch input is in the firstintensity stage (e.g., with the determining unit 1312).

In some embodiments, the processing unit 1308 is configured to,subsequent to determining that the touch input remains in the secondintensity stage, determine a third intensity applied by the touch inputon the touch-sensitive surface unit 1304 (e.g., with the determiningunit 1312), wherein the third intensity is distinct from the secondintensity; and, in accordance with a determination that the thirdintensity satisfies a stage activation threshold for the third intensitystage, distinct from the stage activation intensity threshold for thesecond intensity stage, determine that the touch input is in the thirdintensity stage (e.g., with the determining unit 1312).

In some embodiments, the processing unit 1308 is configured to,subsequent to determining that the touch input is in the third intensitystage, determine a fourth intensity applied by the touch input on thetouch-sensitive surface unit 1304 (e.g., with the determining unit1312), wherein the fourth intensity is distinct from the thirdintensity; and, in accordance with a determination that the fourthintensity satisfies a stage release intensity threshold for the thirdintensity stage, distinct from the stage activation threshold for thethird intensity stage, determine that the touch input is in the secondintensity stage (e.g., with the determining unit 1312).

In some embodiments, the processing unit 1308 is configured to identifyan intensity model identifier from a plurality of predefined intensitymodel identifiers (e.g., with the identifying unit 1314), wherein theintensity stage of the touch input is selected from a plurality ofintensity stages that correspond to the identified intensity modelidentifier.

In accordance with some embodiments, FIG. 14 shows a functional blockdiagram of electronic device 1400 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software,firmware, or a combination thereof to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 14 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 14, electronic device 1400 includes display unit 1402configured to display a user interface, touch-sensitive surface unit1404 configured to receive contacts, one or more sensor units 1406configured to detect intensity of contacts with the touch-sensitivesurface unit 1404, one or more tactile output units 1408; and processingunit 1408 coupled with display unit 1402, the touch-sensitive surfaceunit 1404, the one or more sensor units 1406, and the one or moretactile output units 1408. In some embodiments, the processing unit 1410includes: detecting unit 1412, determining unit 1414, identifying unit1416, generating unit 1418, selecting unit 1420, sending unit 1422,receiving unit 1424, processing unit 1426, and foregoing unit 1428.

The processing unit 1410 is configured to: detect a first touch input onthe touch-sensitive surface unit 1404 (e.g., with the determining unit1414 and/or the touch-sensitive surface unit 1404); in response todetecting the first touch input on the touch-sensitive surface unit1404, determine a first intensity applied by the first touch input onthe touch-sensitive surface unit 1404 (e.g., with the determining unit1414, the touch-sensitive surface unit 1404, and/or the sensor units1406); identify a first intensity model identifier from a plurality ofpredefined intensity model identifiers (e.g., with the identifying unit1416); in accordance with the first intensity applied by the first touchinput on the touch-sensitive surface unit 1404 and one or morethresholds associated with the first intensity model identifier,determine a first touch characterization parameter (e.g., with thedetermining unit 1414); and, subsequent to determining the first touchcharacterization parameter, send first touch information to a firstsoftware application (e.g., with the sending unit 1422), wherein thefirst touch information includes the first intensity model identifierand the first touch characterization parameter.

In some embodiments, the processing unit 1410 is configured to, inresponse to detecting the first touch input on the touch-sensitivesurface unit 1404, generate a tactile output in accordance with theintensity applied by the first touch input on the touch-sensitivesurface unit 1404 and the one or more thresholds associated with thefirst intensity model identifier (e.g., with the generating unit 1418and/or the tactile output units 1408).

In some embodiments, the processing unit 1410 is configured to: receiveone or more instructions from the first software application to generatea tactile output (e.g., with the receiving unit 1424); and, in responseto receiving the one or more instructions from the software application,generate the tactile output in accordance with the one or moreinstructions from the software application (e.g., with the generatingunit 1418 and/or the tactile output units 1408).

In some embodiments, the processing unit 1410 is configured to, whilecontinuing to detect the first touch input on the touch-sensitivesurface unit 1404: receive one or more instructions to use an intensitymodel that corresponds to a second intensity model identifier that isdistinct from the first intensity model identifier (e.g., with thereceiving unit 1424); and, subsequent to receiving the one or moreinstructions to use the intensity model that corresponds to the secondintensity model identifier: determine a second intensity applied by thefirst touch input on the touch-sensitive surface unit 1404 (e.g., withthe determining unit 1414, the touch-sensitive surface unit 1404, and/orthe sensor units 1406); and process the first touch input based on thesecond intensity model identifier (e.g., with the processing unit 1426).

In some embodiments, the processing unit 1410 is configured to, whilecontinuing to detect the first touch input on the touch-sensitivesurface unit 1404: subsequent to processing the first touch input basedon the second intensity model identifier, receive one or moreinstructions to use an intensity model that corresponds to a thirdintensity model identifier that is distinct from the first intensitymodel identifier and the second intensity model identifier (e.g., withthe receiving unit 1424); and, subsequent to receiving the one or moreinstructions to use the intensity model that corresponds to the thirdintensity model identifier: determine a third intensity applied by thefirst touch input on the touch-sensitive surface unit 1404 (e.g., withthe determining unit 1414, the touch-sensitive surface unit 1404, and/orthe sensor units 1406); and process the first touch input based on thethird intensity model identifier (e.g., with the processing unit 1426).

In some embodiments, the processing unit 1410 is configured to:determine that the first touch input has ceased to be detected on thetouch-sensitive surface unit 1404 (e.g., with the determining unit1414); and, subsequent to determining that the first touch input hasceased to be detected on the touch-sensitive surface unit 1404: detect asecond touch input on the touch-sensitive surface unit 1404 that isseparate from the first touch input (e.g., with the detecting unit 1412and/or the touch-sensitive surface unit 1404); and process the secondtouch input based on the first intensity model identifier (e.g., withthe processing unit 1426).

In some embodiments, processing the touch input based on the secondintensity model identifier includes: in accordance with the secondintensity applied by the touch input on the touch-sensitive surface unit1404 and one or more thresholds associated with the second intensitymodel identifier, determining a second touch characterization parameter(e.g., with the determining unit 1414), wherein the second touchcharacterization parameter is distinct from the first touchcharacterization parameter; and, subsequent to determining the secondtouch characterization parameter, sending second touch information tothe first software application (e.g., with the sending unit 1422),wherein the second touch information includes the second intensity modelidentifier and the second touch characterization parameter.

In some embodiments, processing the touch input based on the secondintensity model identifier includes: foregoing generation of a tactileoutput in accordance with the second intensity failing to satisfying theone or more thresholds associated with the second intensity modelidentifier (e.g., with the foregoing unit 1428, the generating unit1418, and/or the tactile output units 1408), wherein the electronicdevice is configured to generate a tactile output in accordance with thesecond intensity satisfying at least one of the one or more thresholdsassociated with the first intensity model identifier.

In some embodiments, the processing unit 1410 is configured to:subsequent to detecting the first touch input, receive one or moreinstructions to use an intensity model that corresponds to a secondintensity model identifier that is distinct from the first intensitymodel identifier (e.g., with the receiving unit 1424); and, subsequentto receiving the one or more instructions to use the intensity modelthat corresponds to the second intensity model identifier: detect asecond touch input on the touch-sensitive surface unit 1404 (e.g., withthe detecting unit 142 and/or the touch-sensitive surface unit 1404); inresponse to detecting the second touch input on the touch-sensitivesurface unit 1404, determine a second intensity applied by the secondtouch input on the touch-sensitive surface unit 1404 (e.g., with thedetermining unit 1414, the touch-sensitive surface unit 1404, and/or thesensor units 1406); and process the second touch input based on thesecond intensity model identifier (e.g., with the processing unit 1426).

In some embodiments, the processing unit 1410 is configured to send astream of intensity events to the first software application (e.g., withthe sending unit 1422), each intensity event corresponding to anintensity applied by the touch input at a corresponding time.

In some embodiments, the processing unit 1410 is configured to select aset of thresholds, from a plurality of sets of thresholds, in accordancewith the identified intensity model identifier (e.g., with the selectingunit 1420), and determine the touch characterization parameter inaccordance with the selected set of thresholds (e.g., with thedetermining unit 1414).

In some embodiments, the processing unit 1410 is configured to identifyone or more intensity ranges, in accordance with the identifiedintensity model identifier (e.g., with the identifying unit 1416), anddetermine a touch characterization parameter in accordance with the oneor more identified intensity ranges (e.g., with the determining unit1414).

In some embodiments, identifying the first intensity model identifierincludes identifying the first software application as corresponding tothe touch input and identifying an intensity model identifier registeredby the first software application as the first intensity modelidentifier (e.g., with the identifying unit 1416).

In accordance with some embodiments, FIG. 15 shows a functional blockdiagram of electronic device 1500 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software,firmware, or a combination thereof to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 15 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 15, electronic device 1500 includes display unit 1502configured to display a user interface, touch-sensitive surface unit1504 configured to receive contacts, one or more sensor units 1506configured to detect intensity of contacts with the touch-sensitivesurface unit 1504, one or more tactile output units 1508; and processingunit 1508 coupled with display unit 1502, the touch-sensitive surfaceunit 1504, the one or more sensor units 1506, and the one or moretactile output units 1508. In some embodiments, the processing unit 1510includes: detecting unit 1512, identifying unit 1514, determining unit1516, sending unit 1518, generating unit 1520, selecting unit 1522,foregoing unit 1524, and display enabling unit 1526.

The processing unit 1510 is configured to: detect a first touch input ona first touch region of the touch-sensitive surface unit 1504 (e.g.,with the detecting unit 1512 and/or the touch-sensitive surface unit1504); identify a first intensity model identifier, associated with thefirst touch region of the touch-sensitive surface unit 1504, from aplurality of predefined intensity model identifiers (e.g., with theidentifying unit 1514); in response to detecting the first touch inputon the first touch region of the touch-sensitive surface unit 1504:determine a first intensity applied by the first touch input on thefirst touch region of the touch-sensitive surface unit 1504 (e.g., withthe determining unit 1516, the touch-sensitive surface unit 1504, and/orthe sensor units 1506); in accordance with the first intensity appliedby the first touch input on the touch-sensitive surface unit 1504 andone or more thresholds associated with the first intensity modelidentifier, determine a first touch characterization parameter (e.g.,with the determining unit 1516); and, subsequent to determining thefirst touch characterization parameter, send first touch information tothe first software application (e.g., with the sending unit 1518),wherein the first touch information includes the first touchcharacterization parameter; detect a second touch input on a secondtouch region of the touch-sensitive surface unit 1504 (e.g., with thedetecting unit 1512 and/or the touch-sensitive surface unit 1504),wherein the second touch region of the touch-sensitive surface unit 1504is distinct from the first touch region of the touch-sensitive surfaceunit 1504; identify a second intensity model identifier, associated withthe second touch region of the touch-sensitive surface unit 1504, fromthe plurality of predefined intensity model identifiers (e.g., with theidentifying unit 1514); and, in response to detecting the second touchinput on the second touch region of the touch-sensitive surface unit1504: determine a second intensity applied by the second touch input onthe second touch region of the touch-sensitive surface unit 1504 (e.g.,with the determining unit 1516, the touch-sensitive surface unit 1504,and/or the sensor units 1506); in accordance with the second intensityapplied by the second touch input on the touch-sensitive surface unit1504 and one or more thresholds associated with the second intensitymodel identifier, determine a second touch characterization parameter(e.g., with the determining unit 1516); and, subsequent to determiningthe second touch characterization parameter, send second touchinformation to the first software application (e.g., with the sendingunit 1518), wherein the second touch information includes the secondtouch characterization parameter.

In some embodiments, the processing unit 1510 is configured to, prior todetecting the first touch input, enable display of a user interface thatincludes a first display region and a second display region (e.g., withthe display enabling unit 1526 and/or the display unit 1502), whereinthe first display region corresponds to the first touch region and thesecond display region corresponds to the second touch region.

In some embodiments, the first display region corresponds to a userinterface of the first software application and the second displayregion corresponds to a user interface of a second software application.

In some embodiments, the processing unit 1510 is configured to: inresponse to detecting the first touch input on the first touch region ofthe touch-sensitive surface unit 1504, generate a first tactile outputin accordance with the first intensity applied by the first touch inputon the touch-sensitive surface unit 1504 and the one or more thresholdsassociated with the first intensity model identifier (e.g., with thegenerating unit 1520 and/or the tactile output units 1508); and, inresponse to detecting the second touch input on the second touch regionof the touch-sensitive surface unit 1504, generate a second tactileoutput in accordance with the second intensity applied by the secondtouch input on the touch-sensitive surface unit 1504 and the one or morethresholds associated with the second intensity model identifier (e.g.,with the generating unit 1520 and/or the tactile output units 1508).

In some embodiments, at least a portion of the first touch regionoverlaps with at least a portion of the second touch region.

In some embodiments, the processing unit 1510 is configured to: detect athird touch input on an overlapping touch region, of the touch-sensitivesurface unit 1504, that corresponds to an overlap of the first touchregion and the second touch region (e.g., with the detecting unit 1512and/or the touch-sensitive surface unit 1504); select an intensity modelidentifier between the first intensity model identifier and the secondintensity model identifier for the overlapping touch region (e.g., withthe selecting unit 1522); and, in response to detecting the third touchinput on the overlapping touch region: determine a third intensityapplied by the third touch input on the overlapping touch region of thetouch-sensitive surface unit 1504 (e.g., with the determining unit 1516,the touch-sensitive surface unit 1504, and/or the sensor units 1506); inaccordance with the third intensity applied by the third touch input onthe overlapping touch region of the touch-sensitive surface unit 1504and one or more thresholds associated with the selected intensity modelidentifier, determine a third touch characterization parameter (e.g.,with the determining unit 1516); and, subsequent to determining thethird touch characterization parameter, send third touch information tothe first software application (e.g., with the sending unit 1518),wherein the third touch information includes the third touchcharacterization parameter.

In some embodiments, both the first intensity model identifier and thesecond intensity model identifier are associated with prioritiesapplicable to the overlapping touch region, and the intensity modelidentifier is selected based on the priorities of the first intensitymodel identifier and the second intensity model identifier for theoverlapping touch region.

In some embodiments, the processing unit 1510 is configured to foregodetermination of a touch characterization parameter in accordance withthe third intensity and one or more thresholds associated with anintensity model identifier that has not been selected between the firstintensity model identifier and the second intensity model identifier(e.g., with the foregoing unit 1524 and/or the determining unit 1516).

In some embodiments, the first intensity model identifier has beenselected for the overlapping region (e.g., by the selecting unit 1518),and the processing unit 1510 is configured to: subsequent to detectingthe third touch input, detect a fourth touch input on the overlappingregion of the touch-sensitive surface unit 1504 (e.g., with thedetecting unit 1512 and/or the touch-sensitive surface unit 1504); andselect the second intensity model identifier for the overlapping touchregion (e.g., with the selecting unit 1522); and, in response todetecting the fourth touch input on the overlapping touch region:determine a fourth intensity applied by the fourth touch input on theoverlapping touch region of the touch-sensitive surface unit 1504 (e.g.,with the determining unit 1516, the touch-sensitive surface unit 1504,and/or the sensor units 1506); in accordance with the fourth intensityapplied by the fourth touch input on the overlapping touch region of thetouch-sensitive surface unit 1504 and the one or more thresholdsassociated with the second intensity model identifier, determine afourth touch characterization parameter (e.g., with the determining unit1516); and, subsequent to determining the touch characterizationparameter, send fourth touch information to the first softwareapplication (e.g., with the sending unit 1518), wherein the fourth touchinformation includes the fourth touch characterization parameter.

In accordance with some embodiments, FIG. 16 shows a functional blockdiagram of electronic device 1600 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software,firmware, or a combination thereof to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 16 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 16, electronic device 1600 includes display unit 1602configured to display a user interface, touch-sensitive surface unit1604 configured to receive contacts, one or more sensor units 1606configured to detect intensity of contacts with the touch-sensitivesurface unit 1604, one or more tactile output units 1608; and processingunit 1608 coupled with display unit 1602, the touch-sensitive surfaceunit 1604, the one or more sensor units 1606, and the one or moretactile output units 1608. In some embodiments, the processing unit 1610includes: detecting unit 1612, processing unit 1614, providing unit1616, determining unit 1618, identifying unit 1620, generating unit1622, display enabling unit 1624, selecting unit 1626, updating unit1628, and foregoing unit 1630.

The processing unit 1610 is configured to: detect a touch input on thetouch-sensitive surface unit 1604 (e.g., with the detecting unit 1612and/or the touch-sensitive surface unit 1604); and, in response todetecting the touch input: in accordance with a determination that thetouch input is at a location on the touch-sensitive surface unit 1604that is associated with a first intensity model of a plurality ofdifferent intensity models, process the touch input in accordance withan intensity applied by the touch input on the touch-sensitive surfaceunit 1604 and the first intensity model (e.g., with the processing unit1614); and, in accordance with a determination that the touch input isat a location on the touch-sensitive surface unit 1604 that isassociated with a second intensity model different from the firstintensity model, process the touch input in accordance with an intensityapplied by the touch input on the touch-sensitive surface unit 1604 andthe second intensity model (e.g., with the processing unit 1614).

In some embodiments, processing the touch input in accordance with anintensity applied by the touch input and the first intensity modelincludes providing contact intensity information to a user-interfaceapplication without generating a tactile output for the touch input(e.g., with the providing unit 1616); and processing the touch input inaccordance with an intensity applied by the touch input and the secondintensity model includes conditionally generating a tactile output forthe touch input (e.g., with the generating unit 1622 and/or the tactileoutput units 1608).

In some embodiments, during the touch input, the electronic device 1600detects application of a respective intensity on the touch-sensitivesurface unit 1604 that is attributed to the touch input (e.g., with thedetecting unit 1612, the touch-sensitive surface unit 1604, thedetermining unit 1618, and/or the sensor units 1606). Processing thetouch input in accordance with an intensity applied by the touch inputon the touch-sensitive surface unit 1604 and the first intensity modelincludes providing, to a user interface application, first touch inputstate information that indicates that the touch input has applied therespective intensity on the touch-sensitive surface unit 1604 withoutgenerating a tactile output for the touch input (e.g., with theproviding unit 1616); and processing the touch input in accordance withan intensity applied by the touch input on the touch-sensitive surfaceunit 1604 and the second intensity model includes providing, to a userinterface application, second touch input state information thatindicates that the touch input has applied the respective intensity onthe touch-sensitive surface unit 1604 (e.g., with the providing unit1616) and generating a tactile output for the touch input (e.g., withthe generating unit 1622 and/or the tactile output unit 1608).

In some embodiments, the first touch input state information includes acontinuously variable representation of intensity of the touch input;and the second touch input state information includes an indication thatthe intensity of the touch input is within a range of values thatcorrespond to activation of a user interface element that corresponds tothe location on the touch-sensitive surface unit 1604 that is associatedwith the second intensity model.

In some embodiments, the processing unit 1610 is configured to: detect afirst touch input on a first touch region of the touch-sensitive surfaceunit 1604 (e.g., with the detecting unit 1612 and/or the touch-sensitivesurface unit 1604); identify a first intensity model identifier,associated with the first touch region of the touch-sensitive surfaceunit 1604, from a plurality of predefined intensity model identifiers(e.g., with the identifying unit 1620); in response to detecting thefirst touch input on the first touch region of the touch-sensitivesurface unit 1604: determine a first intensity applied by the firsttouch input on the first touch region of the touch-sensitive surfaceunit 1604 (e.g., with the determining unit 1618, the touch-sensitivesurface unit 1604, and/or the sensor units 1606); and, in accordancewith the first intensity applied by the first touch input on thetouch-sensitive surface unit 1604 and the first intensity modelidentifier: process the first touch input (e.g., with the processingunit 1614); and forego generation of a tactile output for the firsttouch input (e.g., with the foregoing unit 1630, the generating unit1622, and/or the tactile output units 1608); detect a second touch inputon a second touch region of the touch-sensitive surface unit 1604 (e.g.,with the detecting unit 1612 and/or the touch-sensitive surface unit1604), wherein the second touch region of the touch-sensitive surfaceunit 1604 is distinct from the first touch region of the touch-sensitivesurface unit 1604; identify a second intensity model identifier,associated with the second touch region of the touch-sensitive surfaceunit 1604, from the plurality of predefined intensity model identifiers(e.g., with the identifying unit 1620); and, in response to detectingthe second touch input on the second touch region of the touch-sensitivesurface unit 1604: determine a second intensity applied by the secondtouch input on the second touch region of the touch-sensitive surfaceunit 1604 (e.g., with the determining unit 1618, the touch-sensitivesurface unit 1604, and/or the sensor units 1606); and, in accordancewith the second intensity applied by the second touch input on thetouch-sensitive surface unit 1604 and the second intensity modelidentifier: process the second touch input (e.g., with the processingunit 1614); and conditionally generate a tactile output for the secondtouch input in accordance with the second touch input and one or moremodel parameters associated with the second intensity model identifier(e.g., with the generating unit 1622 and/or the tactile output units1608).

In some embodiments, conditionally generating the tactile output for thesecond touch input includes foregoing generation of the tactile outputfor the second touch input in accordance with a determination that thesecond touch input fails to satisfy one or more criteria correspondingto the one or more model parameters associated with the second intensitymodel identifier (e.g., with the foregoing unit 1630, the generatingunit 1622, and/or the tactile output units 1608).

In some embodiments, the first intensity model identifier indicates thatno tactile output is to be generated for a touch input on the firsttouch region of the touch-sensitive surface unit 1604.

In some embodiments, the electronic device 1600 stores a first softwareapplication, and the first user input and the second user input aredetected (e.g., with the detecting unit 1612 and/or the touch-sensitivesurface unit 1604) while the electronic device 1600 displays a userinterface of the first software application (e.g., with the displayenabling unit 1624 and/or the display unit 1602).

In some embodiments, the user interface of the first softwareapplication includes a handwriting input tool region.

In some embodiments, the handwriting input tool region includes aplurality of selection regions, and at least a respective selectionregion of the one or more selection regions corresponds to the secondtouch region of the touch-sensitive surface unit 1604.

In some embodiments, the handwriting input tool region includes ahandwriting input region, and the handwriting input region correspondsto the first touch region of the touch-sensitive surface unit 1604.

In some embodiments, the processing unit 1610 is configured to, inresponse to detecting the first touch input on the first touch region ofthe touch-sensitive surface unit 1604, enable display of one or moregraphical elements in the handwriting input region in accordance withthe first touch input (e.g., with the display enabling unit 1624 and/orthe display unit 1602).

In some embodiments, the processing unit 1610 is configured to, inresponse to detecting the first touch input on the first touch region ofthe touch-sensitive surface unit 1604, enable display of a plurality ofgroups of characters in the plurality of selection regions (e.g., withthe display enabling unit 1624 and/or the display unit 1602), wherein arespective group of characters is selected based on the one or moredisplayed graphical elements in the handwriting input region, and therespective group of characters is displayed in a respective selectionregion; and, in response to detecting the second user input on thesecond touch region of the touch-sensitive surface unit 1604 at alocation that corresponds to the respective selection region, select therespective group of characters (e.g., with the selecting unit 1626).

In some embodiments, processing the first touch input detected on thefirst touch region of the touch-sensitive surface unit 1604 includes:determining that the first touch input has satisfied a drawing intensitythreshold (e.g., with the determining unit 1618); and, in accordancewith a determination that the first touch input has exceeded the drawingintensity threshold, enabling display of the one or more graphicalelements in the handwriting input region in accordance with the firsttouch input (e.g., with the display enabling unit 1624 and/or thedisplay unit 1602), wherein a first terminal region of a respectivegraphical element of the one or more graphical elements corresponds to alocation where the first touch input has been determined to satisfy thedrawing intensity threshold.

In some embodiments, processing the first touch input detected on thefirst touch region of the touch-sensitive surface unit 1604 includes:while continuing to detect the first touch input on the touch-sensitivesurface unit 1604, continue to update the one or more graphical elementsin the handwriting input region in accordance with the first touch inputregardless of whether the first touch input has ceased to satisfy thedrawing intensity threshold (e.g., with the updating unit 1628).

In some embodiments, processing the first touch input detected on thefirst touch region of the touch-sensitive surface unit 1604 includes:detecting that the first touch input has ceased to be detected on thetouch-sensitive surface unit 1604 (e.g., with the detecting unit 1612and/or the touch-sensitive surface unit 1604) and detecting that thefirst touch input has resumed contact with the touch-sensitive surfaceunit 1604 within a predefined time interval subsequent to the firsttouch input ceasing to be detected on the touch-sensitive surface unit1604 (e.g., with the detecting unit 1612 and/or the touch-sensitivesurface unit 1604); and, in response to the first touch input resumingcontact with the touch-sensitive surface unit 1604 within the predefinedtime interval subsequent to the first touch input ceasing to be detectedon the touch-sensitive surface unit 1604, updating the one or moregraphical elements in the handwriting input region in accordance withthe first touch input (e.g., with the updating unit 1628).

In accordance with some embodiments, FIG. 17 shows a functional blockdiagram of electronic device 1700 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software,firmware, or a combination thereof to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 17 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 17, electronic device 1700 includes display unit 1702configured to display a user interface, touch-sensitive surface unit1704 configured to receive contacts, one or more sensor units 1706configured to detect intensity of contacts with the touch-sensitivesurface unit 1704; and processing unit 1708 coupled with display unit1702, the touch-sensitive surface unit 1704 and the one or more sensorunits 1706. In some embodiments, the processing unit 1710 includes:display enabling unit 1710, detecting unit 1712, processing unit 1714,activating unit 1716, sending unit 1718, and updating unit 1720.

The processing unit 1710 is configured to: enable display of a userinterface that includes two or more display regions, including a firstdisplay region and a second display region (e.g., with the displayenabling unit 1710 and/or the display unit 1702); and, while displayingthe user interface: detect a first intensity applied by a touch input ata first location on the touch-sensitive surface unit 1704 thatcorresponds to the first display region (e.g., with the detecting unit1712, the touch-sensitive surface unit 1704, and/or the sensor units1706); detect a movement of the touch input across the touch-sensitivesurface unit 1704 from the first location on the touch-sensitive surfaceunit 1704 to a second location on the touch-sensitive surface unit 1704that corresponds to the second display region (e.g., with the detectingunit 1712 and/or the touch-sensitive surface unit 1704); after detectingthe movement of the touch input from the first location on thetouch-sensitive surface unit 1704 to the second location on thetouch-sensitive surface unit 1704, detect a second intensity applied bythe touch input at the second location on the touch-sensitive surfaceunit 1704 (e.g., with the detecting unit 1712, the touch-sensitivesurface unit 1704, and/or the sensor units 1706); and, in response todetecting the second intensity applied by the touch input at the secondlocation on the touch-sensitive surface unit 1704: in accordance with adetermination that the first intensity does not satisfy a firstintensity threshold, process the second intensity in accordance with oneor more intensity thresholds associated with the second display region(e.g., with the processing unit 1714); and, in accordance with adetermination that the first intensity satisfies the first intensitythreshold, process the second intensity in accordance with one or moreintensity thresholds associated with the first display region (e.g.,with the processing unit 1714).

In some embodiments, the processing unit 1708 is configured to: sendfirst touch information to the first software application in accordancewith a determination that the first intensity satisfies a reportingintensity threshold that is distinct from the first intensity threshold,the first touch information including one or more touch parameters thatcorrespond to the first intensity (e.g., with the sending unit 1718);and send second touch information to the first software application inaccordance with a determination that the second intensity satisfies thereporting intensity threshold, the second touch information includingone or more touch parameters that correspond to the second intensity(e.g., with the sending unit 1718).

In some embodiments, a first display region of the two or more displayregions is associated with a first software element of the firstsoftware application and a second display region of the two or moredisplay regions is associated with a second software element of thesecond software application.

In some embodiments, in accordance with the determination that the firstintensity does not satisfy the first intensity threshold, the secondtouch information is sent to the second software element without sendingthe second touch information to the first software element (e.g., withthe sending unit 1718).

In some embodiments, in accordance with the determination that the firstintensity satisfies the first intensity threshold, the second touchinformation is sent to the first software element without sending thesecond touch information to the second software element (e.g., with thesending unit 1718).

In some embodiments, in accordance with the determination that the firstintensity satisfies the reporting intensity threshold, the first touchinformation is sent to the first software element (e.g., with thesending unit 1718).

In some embodiments, the processing unit 1708 is configured to, inresponse to detecting the touch input, update the user interface inaccordance with the first touch information using the first softwareapplication prior to the first intensity satisfying the first intensitythreshold (e.g., with the updating unit 1720).

In some embodiments, the first intensity threshold is distinct from anactivation intensity threshold.

In some embodiments, the one or more intensity thresholds associatedwith the first display region are different from the one or moreintensity thresholds associated with the second display region.

In some embodiments, the one or more intensity thresholds associatedwith the first display region are the same as the one or more intensitythresholds associated with the second display region.

In some embodiments, the processing unit 1708 is configured to, inresponse to detecting the first intensity applied by the touch input,process the first intensity in accordance with the one or more intensitythresholds associated with the first display region (e.g., with theprocessing unit 1714).

In some embodiments, processing the first intensity in accordance withthe one or more intensity thresholds associated with the first displayregion includes enabling display of an animation that corresponds to achange in intensity applied by the touch input from a prior intensity tothe first intensity (e.g., with the display enabling unit 1710 and/orthe display unit 1702).

In some embodiments, processing the second intensity in accordance withthe one or more intensity thresholds associated with the first displayregion includes enabling display of an animation that corresponds to achange in intensity applied by the touch input on the touch-sensitivesurface unit 1704 (e.g., with the display enabling unit 1710 and/or thedisplay unit 1702).

In some embodiments, processing the second intensity in accordance withthe one or more intensity thresholds associated with the second displayregion includes enabling display of an animation that corresponds to achange in intensity applied by the touch input on the touch-sensitivesurface unit 1704 (e.g., with the display enabling unit 1710 and/or thedisplay unit 1702).

In some embodiments, the processing unit 1708 is configured to, inresponse to detecting the movement of the touch input from the firstlocation on the touch-sensitive surface unit 1704 to the second locationon the touch-sensitive surface unit 1704, enable display of an animationthat corresponds to a change in intensity applied by the touch input(e.g., with the display enabling unit 1710 and/or the display unit1702).

In some embodiments, processing the second intensity in accordance withone or more intensity thresholds associated with the first displayregion includes activating a first control associated with the firstdisplay region (e.g., with the activating unit 1716).

In some embodiments, processing the second intensity in accordance withone or more intensity thresholds associated with the second displayregion includes activating a second control associated with the seconddisplay region (e.g., with the activating unit 1716).

In accordance with some embodiments, FIG. 18 shows a functional blockdiagram of an electronic device 1800 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 18 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 18, an electronic device 1800 includes a display unit1802 configured to display a user interface for the electronic device, aplurality of input units 1804, including a first input unit that isconfigured to generate tactile outputs in response to inputs, and asecond input unit that is configured to generate tactile outputs. Insome embodiments, one or more of the input units 1804 includes one ormore respective sensor unit(s) 1806. The electronic device 1800 furtherincludes a processing unit 1808 coupled to the display unit 1802, theinput units 1804, and the sensors 1806. In some embodiments, theprocessing unit 1808 includes an input indication receiving unit 1810configured to receive an indication of an input detected by a respectiveinput unit of the plurality of input units, and an information providingunit 1812 configured to provide, in response to receiving the indicationof the input, information describing the input to an application runningon the electronic device that enables the application to react to theinput.

The processing unit 1808 further includes a reaction receiving unit 1814configured to receive a reaction to the input from the application thatindicates that a tactile output is to be generated in response to theinput, and a causing unit 1816 configured to cause, in response toreceiving the reaction to the input from the application, the generationof a respective tactile output. More specifically, in accordance with adetermination (see operation 1114 of method 1100, FIGS. 11A-11C) thatthe reaction was triggered by the first input device, the causing unit1816 causes the respective tactile output to be generated at the firstinput device based on the reaction to the input from the application;and in accordance with a determination (see operation 1116 of method1100, FIGS. 11A-11C) that the reaction was triggered by the second inputdevice, the causing unit 1816 causes the respective tactile output to begenerated at the second input device based on the reaction to the inputfrom the application.

In some embodiments, processing unit 1808 of electronic device 1800 isconfigured to cause the electronic device to perform any of the methodsdescribed above with reference to FIGS. 11A-11C.

In accordance with some embodiments, FIG. 19 shows a functional blockdiagram of an electronic device 1900 configured in accordance with theprinciples of the various described embodiments. The functional blocksof the device are, optionally, implemented by hardware, software, or acombination of hardware and software to carry out the principles of thevarious described embodiments. It is understood by persons of skill inthe art that the functional blocks described in FIG. 19 are, optionally,combined or separated into sub-blocks to implement the principles of thevarious described embodiments. Therefore, the description hereinoptionally supports any possible combination or separation or furtherdefinition of the functional blocks described herein.

As shown in FIG. 19, an electronic device 1900 includes a display unit1902 configured to display a user interface for the electronic device,and one or more input units 1904 configured to generate tactile outputsin response to inputs. In some embodiments, one or more of the inputunits 1904 includes one or more respective sensor unit(s) 1906. Theelectronic device 1900 further includes a processing unit 1908 coupledto display unit 1902, input units 1904, and sensors 1906. In someembodiments, processing unit 1908 includes an input indication receivingunit 1910 configured to receive an indication of an input detected by arespective input unit of the plurality of input units, and aninformation providing unit 1912 configured to provide, in response toreceiving the indication of the input, information describing the inputto an application running on the electronic device that enables theapplication to react to the input.

The processing unit 1908 further includes a reaction receiving unit 1914configured to receive a reaction to the input from the application thatindicates that a tactile output is to be generated in response to theinput, and a causing unit 1916 configured to cause, in response toreceiving the reaction to the input from the application, the generationof a respective tactile output. More specifically, in accordance with adetermination (see operation 1114 of method 1100, FIGS. 11A-11C) thatthe reaction was triggered by the first input device, the causing unit1916 causes the respective tactile output to be generated at the firstinput device based on the reaction to the input from the application;and in accordance with a determination (see operation 1116 of method1100, FIGS. 11A-11C) that the reaction was triggered by the second inputdevice, the causing unit 1916 causes the respective tactile output to begenerated at the second input device based on the reaction to the inputfrom the application.

In some embodiments, processing unit 1908 of electronic device 1900 isconfigured to cause the electronic device to perform any of the methodsdescribed above with reference to FIGS. 11A-11C.

The operations described above with reference to FIGS. 6A-6D, 7A-7D,8A-8C, 9A-9D, 10A-10D, 11A-11C, and 12A-12D are, optionally, implementedby components depicted in FIGS. 1A-1B and/or FIGS. 13-19. For example,as to the operations described above with reference to FIGS. 11A-11C,receiving an indication operation 1104, providing information describingthe input operation 1106, receiving a reaction to the input from theapplication operation 1108, and causing the generation of a tactileoutput operation 1112 are, optionally, implemented by event sorter 170,event recognizer 180, and event handler 190 (FIG. 1B), or by processingunit 1808 (FIG. 18). Event monitor 171 in event sorter 170 detects acontact on touch-sensitive display 112, and event dispatcher module 174delivers the event information to application 136-1. A respective eventrecognizer 180 of application 136-1 compares the event information torespective event definitions 186, and determines whether a first contactat a first location on the touch-sensitive surface corresponds to apredefined event or sub-event, such as selection of an object on a userinterface. When a respective predefined event or sub-event is detected,event recognizer 180 activates an event handler 190 associated with thedetection of the event or sub-event. Event handler 190 optionallyutilizes or calls data updater 176 or object updater 177 to update theapplication internal state 192. In some embodiments, event handler 190accesses a respective GUI updater 178 to update what is displayed by theapplication and/or accesses a tactile output generator 167 to generate atactile output. Similarly, it would be clear to a person having ordinaryskill in the art how other processes can be implemented based on thecomponents depicted in FIGS. 1A-1B.

For another example, as to the operations described above with referenceto FIGS. 12A-12C, receiving an indication operation 1204, providinginformation describing the input operation 1206, receiving a reaction tothe input from the application operation 1208, and causing theperformance of operations 1210 are, optionally, implemented by eventsorter 170, event recognizer 180, and event handler 190 (FIG. 1B), or byprocessing unit 1908 (FIG. 19). Event monitor 171 in event sorter 170detects a contact on touch-sensitive display 112, and event dispatchermodule 174 delivers the event information to application 136-1. Arespective event recognizer 180 of application 136-1 compares the eventinformation to respective event definitions 186, and determines whethera first contact at a first location on the touch-sensitive surfacecorresponds to a predefined event or sub-event, such as selection of anobject on a user interface. When a respective predefined event orsub-event is detected, event recognizer 180 activates an event handler190 associated with the detection of the event or sub-event. Eventhandler 190 optionally utilizes or calls data updater 176 or objectupdater 177 to update the application internal state 192. In someembodiments, event handler 190 accesses a respective GUI updater 178 toupdate what is displayed by the application and/or accesses a tactileoutput generator 167 to generate a tactile output. Similarly, it wouldbe clear to a person having ordinary skill in the art how otherprocesses can be implemented based on the components depicted in FIGS.1A-1B.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best use the invention and variousdescribed embodiments with various modifications as are suited to theparticular use contemplated.

APPENDIX A Exemplary Intensity Models Stage Progression

In some embodiments, each stage of a fluid gesture includes three zoneswith specific rules governing the transition through the zones. Thesezones are “next,” “activated,” and “release.” In some embodiments, astage progresses through the zones in that order given the followingrules:

-   1. Any stage can move to any deeper activated or next stage.-   2. Any stage can move to any shallower activated or release stage    provided the force is below the current stage release zone (and any    shallower zone that is skipped)-   3. Positive transitionProgress describes moving though a next zone.-   4. Negative transitionProgress describes moving through a release    zone.-   5. When in neither a next or release zone, transitionProgress is 0.-   6. stagePressure describes moving through the activated zone.-   7. When the force is outside the activated zone of the current    stage, transitionProgress shall be 0 when below the zone, or 1 when    above the zone.-   8. It is possible to be in both activated and next zones or,    activated and release zones for the same stage. The gestureBehavior    shall define if this case applies to it.

The chart in FIG. 20A shows the stage zones for a pulse gesture. Thethree zones A1, A2, and A3 show the related zones for stage 1. In someembodiments, zone A1 has a stage value of 0 because the gesture istransitioning from stage 0 to stage 1 and is therefore not latched tostage 1 yet.

Intensity Model Identifier: Force

FIG. 20B illustrates a basic one-stage intensity model (and acorresponding gesture). Once latched to stage 1, the progress value isimmediately variable.

Intensity Model Identifier: Gas Pedal

FIG. 20C illustrates a two-stage intensity model (and exemplarygestures) where there is a dead zone between the mouse down and thestart of variable force. In some embodiments, the force band for thestage 1 activated progress is infinitely small.

Intensity Model Identifier: Deep Click

FIG. 20D illustrates a two-stage intensity model (and exemplarygestures) where the second stage is sometimes referred to as “deep.” Insome embodiments, this is similar to camera shutter button. Each stageis surrounded by bands where an animation is prescribed. Additionally,once a device (and/or a gesture input on the device) reaches a “deep”stage, the device (and/or the gesture input on the device) remains inthe “deep” stage until the gesture input ends (e.g. release to take 0).

Intensity Model Identifier: Pulse

FIG. 20E illustrates a two-stage intensity model (and a correspondinggesture) where the second stage is sometimes referred to as “deep.” Thisis similar to the deep gesture. However, it differs from the deepgesture in that the gesture input can toggle back and forth between thedeep and normal stages.

APPENDIX B Cocoa Application Framework

The Cocoa Application Framework (also referred to as the ApplicationKit, or AppKit) is one of the core Cocoa frameworks. It providesfunctionality and associated APIs for applications, including objectsfor graphical user interfaces (GUIs), event-handling mechanisms,application services, and drawing and image composition facilities.

Marking Updated APIs in Headers

New APIs in headers are marked with decorations that include referencesto “10_10_3”:

 NS_AVAILABLE_MAC(10_10_3), NS_AVAILABLE(10_10_3, <iOS Release>),NS_CLASS_AVAILABLE(10_10_3, <iOS Release>), NS_ENUM_AVAILABLE(10_10_3)or sometimes the construct:  #if MAC_OS_X_VERSION_MAX_ALLOWED >=MAC_OS_X_VERSION_ 10_10_3  . . .  #endif

Runtime Version Check

There are several ways to check for new features provided by the Cocoaframeworks at runtime. One is to look for a given new class or methoddynamically, and not use it if not there. Another is to use the globalvariable NSAppKitVersionNumber (or, in Foundation,NSFoundationVersionNumber):

  double NSAppKitVersionNumber; #define NSAppKitVersionNumber10_0 577#define NSAppKitVersionNumber10_1 620 #define NSAppKitVersionNumber10_2663 #define NSAppKitVersionNumber10_3 743 #defineNSAppKitVersionNumber10_4 824 #define NSAppKitVersionNumber10_5 949#define NSAppKitVersionNumber10_6 1038 #define NSAppKitVersionNumber10_71138 #define NSAppKitVersionNumber10_8 1187 #defineNSAppKitVersionNumber10_9 1265 #define NSAppKitVersionNumber10_10 1343

One typical use of this is to floor( ) the value, and check against thevalues provided in NSApplication.h. For instance:

if (floor(NSAppKitVersionNumber) <= NSAppKitVersionNumber10_8) {  /* Ona 10.8.x or earlier system */ } else if (floor(NSAppKitVersionNumber) <=NSAppKitVersionNumber10_9) {  /* On a 10.9 - 10.9.x system */ } else { /* 10.10 or later system */ }

Special cases or situations for version checking are also discussed inthe release notes as appropriate. For instance some individual headersmay also declare the versions numbers for NSAppKitVersionNumber wheresome bug fix or functionality is available in a given update, forexample:

#define NSAppKitVersionWithSuchAndSuchBadBugFix 1138.42

Backward Compatibility

One backward compatibility mechanism that is occasionally used in theframeworks is to check for the version of the system an application wasbuilt against, and if an older system, modify the behavior to be morecompatible. This is done in cases where bad incompatibility problems arepredicted or discovered; and most of these are listed below in thesenotes.

Typically we detect where an application was built by looking at theversion of the System, Cocoa, AppKit, or Foundation frameworks theapplication was linked against. Thus, as a result of relinking yourapplication against the latest SDK, you might notice differentbehaviors, some of which might cause incompatibilities. In these casesbecause the application is being rebuilt, we expect you to address theseissues at the same time as well. For this reason, if you are doing asmall incremental update of your application to address a few bugs, it'susually best to continue building on the same build environment andlibraries used originally.

In some cases, we provide defaults (preferences) settings which can beused to get the old or new behavior, independent of what system anapplication was built against. Often these preferences are provided fordebugging purposes only; in some cases the preferences can be used toglobally modify the behavior of an application by registering the values(do it somewhere very early, with -[NSUserDefaults registerDefaults:]).

Pressure Events

Some Apple trackpads have the ability to report pressure. Like therotation and magnification gestures that came before it, the pressuregesture is reported as a new event type, NSEventTypePressure, and a newresponder method -(void)pressureChangeWithEvent:(NSEvent*).

Note: The new NSEventTypePressure events are only available in 64 bit.@property (readonly) NSEventPhase phase NS_AVAILABLE_MAC (10_7);

NSEventTypePressure is a fluid gesture. And like all fluid gestures, ithas a phase that describes the sequence of the pressure gesture stream.

@property (readonly) NSInteger stage NS_AVAILABLE_MAC (10_10_3);

A pressure gesture can go through multiple stages.

Stage 0: The lowest stage. Generally this means the user is applyingless pressure than what is required to get a mouse down. Effectively youonly see one stage 0 pressure event per stream when the gesture ends.Stage 1: Equivalent to a mouse down. NSEventTypePressure events are notposted until the user applies enough pressure that the trackpad willalso issue a mouse down.Stage 2: Equivalent to a force click. The user has applied significantadditional pressure greater than what is needed for a mouse down. Stage2 should generally be used as the trigger for an additional action. Forexample, lookup is performed when the pressure stream transitions tostage 2.Note: Generally, the trackpad will actuate as the gesture transitionsacross stages.Note: It is possible for stage to increase or decrease multiple integervalues per change. For example, a quick removal of the user's fingerswhile at stage 2 may cause stage to transition to 0 without a pressureevent with stage 1.@property (readonly) float pressure;

The pressure on the trackpad for the current stage. The range for thisvalue is [0,1]. Each stage has a pressure curve appropriate for thatstage. That is, pressure may change from [0,1] for stage 1. Andlikewise, [0,1] for stage 2.

Note: Pressure of only one stage should be used. Carefully consider theuse case. If variable input is useful for all cases, then use thepressure during stage 1 as this is the most comfortable range for theuser. If variable input is useful only in rare situations where the userinput must not be ambiguous with a regular mouse click, then use stage2. Though, generally, for such cases, use stage 2 to denote this inputand ignore stage 2 pressure. Do not attempt to combine stage 1 and stage2 pressure to get a larger range. Doing so will cause undue stress onthe user's fingers.Note: pressure is not appropriate for weight measurements.@property (readonly) CGFloat stageTransition NS_AVAILABLE_MAC (10_10_3);

The animation value for stage transitions. Positive stageTransitiondescribes approaching the next stage of the pressure gesture. NegativestageTransition describes approaching release of the current stage. Forexample, as the user approaches stage 2, stageTransition will increasetowards 1. The moment the gesture transitions to stage 2,stageTransition immediately return to 0 and will decrease towards −1 asthe user releases pressure on the trackpad until the gesture transitionsto stage 1 again. Generally, only the positive stage transition valuesare animated.

Note: stageTransition doesn't match the pressure curve. There are bandswhere the pressure may change but the stageTransition remains at 0.stageTransition remains at 0 until the gesture is very near a stagetransition.

  @property (readonly) NSEventMask associatedEventsMaskNS_AVAILABLE_MAC(10_10_3);

This property makes it possible to determine on the mouse down ifpressure should be expected from the input device. Since pressure andmouse events are independent streams, you sometimes need to make adecision (for example, a starting pressure) immediately on mouse down.

Example

   if (event.associatedEventMask & NSEventMaskPressure) {  self.pressure = 0; // Pressure events are coming!  } else if(event.subtype == NSTabletPointEventSubtype) {   self.pressure =event.pressure; // tablets embed pressure in the mouse event.  } else {  self.pressure = 1; // This device does not have pressure. Default tofull pressure  }

Getting Pressure Events

There are 3 ways to get pressure events.

1. Override the NSResponder method:-(void)pressureChangeWithEvent:(NSEvent*)event;2. In a tracking loop, add NSEventMaskPressure to the eventMask.

 NSEventMask eventMask = NSLeftMouseDraggedMask | NSLeftMouseUpMask |NSEventMaskPressure;  [self.window trackEventsMatchingMask:eventMasktimeout:NSEventDurationForever mode:NSEventTrackingRunLoopModehandler:{circumflex over ( )}(NSEvent *event, BOOL *stop) {   if(event.type == NSEventTypePressure) {    // yay, pressure!   } else if(event.type == NSLeftMouseUpMask) {    *stop == YES;   }  }]; -OR- NSEvent *event = [self.windownextEventMatchingMask:NSLeftMouseDraggedMask | NSLeftMouseUpMask |NSEventMaskPressure];3. In a NSGestureRecognizer subclass by overriding the method:-(void)pressureChangeWithEvent:(NSEvent*)event;

Spring Loaded Drag & Drop

The Finder spring loading feature has been expanded and adopted in moreplaces. All application windows will spring forward automatically. Tabswill activate themselves in response to spring loading while segmentedcontrols and buttons can be configured to respond to spring loading.

Spring loading is triggered by hovering over a spring loading capabletarget. The length of the hover delay is controlled via theAccessibility Mouse & Trackpad system preference pane.

Using a pressure sensitive trackpad, the user can also trigger springloading by pressing harder on the trackpad during a drag. This is akinto a nested click inside of a drag. During a drag, the user can pressharder. The trackpad will actuate, arming the spring loading under thecursor. When the user relaxes just enough to release the nested click,the trackpad will actuate and trigger the spring loading withoutdropping the drag. Once the user triggers spring loading via a pressuresensitive trackpad, the hover feature is disabled until the user startsa new drag & drop operation.

Spring Loaded Controls

Segmented controls and buttons can be configured to send their action inresponse to the user dragging an item. Set springLoaded to YES and theuser will be able to interact with the control via force clicking orhovering during a drag.

  @property (getter=isSpringLoaded) BOOL springLoadedNS_AVAILABLE_MAC(10_10_3); // sends action on force-click or extendedhover while dragging. Defaults to NO.

Spring Loaded Tab View

Tab views and tab view controllers now change selection in response toforce click or extended hover while the user is dragging an item.

Accelerator Buttons

There are new button types for pressure sensitive trackpads, referred toas “accelerator” buttons. These act like continuous-mode push buttons inthat applications generally respond to them while they are held down,and then stop when they are released. Their main feature is the abilityto interpret variable pressure, allowing the user to directly controlthe speed of the related action by changing the pressure they apply.This is intended to be used for things like controlling the speed offast forward and rewind for media playback, the advancement speed forweek view in calendar, or the zoom speed in maps.

There are two types of accelerator buttons:

-   -   A “regular” type where fine-grained precision is desirable, and        the range of values is a floating point number. Here the        button's value is 0 when not pressed, and ranges from [1 . .        . 2) when pressed.    -   A “multi-level” type with a configurable number of explicit        levels (up to 5). These buttons also present a value of 0 when        not pressed, but have an integer value from [1 . . . N] to        indicate the discrete acceleration levels.

typedef NS_ENUM(NSUInteger, NSButtonType) {  NSAcceleratorButtonNS_ENUM_AVAILABLE_MAC(10_10_3) = 8,  NSMultiLevelAcceleratorButtonNS_ENUM_AVAILABLE_MAC(10_10_3) = 9, } @property NSInteger maxStateNS_AVAILABLE_MAC(10_10_3); // Configures the maximum allowed state forNSMultiLevelAcceleratorButton buttons, allowed values range from [1,5].

The new button types are variants of the standardNSMomentaryLightButton, and work with a variety of bezel styles. Asimple call to setButtonType is all that's necessary to use them—thiswill properly configure relevant aspects of the button. Acceleratorbuttons behave differently than standard NSButtons in a variety of ways.Beyond the additional API above, here are the major behavioraldistinctions:

-   -   Accelerator buttons do not automatically advance their state        when clicked, but will change value and send action messages        repeatedly while interacting with the user. On mouse up, they        reset their value to 0 and send a final action message.    -   The value of a multi-level accelerator button can be explicitly        set in order to suppress lower levels of the accelerated range.        This is used, for example, when a video is already fast        forwarding at 4×: we provide no acceleration messages or        feedback until the 8× level is reached.    -   Accelerator buttons allow a greater range of values: [0,1.99999        . . . ] or [0,5] compared to the standard on/off/mixed for other        buttons.    -   The floatValue and doubleValue are used to convey pressure        levels for regular accelerator buttons.    -   Accelerator buttons do not support mixed state.

All accelerator buttons will carry a state of 0 when not pressed, or 1when pressed/clicked normally. The state (and value) will rise above 1when pressed harder. For fine grained control over speed,NSButtonTypeAccelerator exposes a doubleValue ranging from 1 wheninitially pressed to 1.999999 (or so) when fully pressed. This designallows the doubleValue and integerValue to “match”, and avoids having toexpose a separate property for the fractional part of the value.

An NSMultiLevelAcceleratorButton provides a configurable number ofdistinct pressure levels, with tactile feedback as the user reaches eachone. Clients configure the number of discrete levels by changing the newmaxState property. It is always 1 for other button types, but defaultsto 2 for multi-level accelerator buttons. Values outside the range [2,5]will be pinned and cause a warning to be logged. Values above 1 will addadditional levels to the button, with a light actuation, on capablehardware, when each one is reached. The control takes on theseinteger-valued states during interaction with the user, and sends anaction message whenever they change. As an aside, note that thepressures needed to reach a given level remain the same, regardless ofthe maxState value. In other words, the pressure thresholds for theallowed levels are not evenly spread across the available pressurerange.

Applications respond to accelerator buttons using the traditional actionhandlers. They read the control's state/integerValue or doubleValue, andconfigure the speed of the related operation to match. The applicationis responsible for translating the floating point [0,1.99999] or integer[0,5] values into an appropriate speed for their specific use case.

Accelerator Segmented Controls

Segmented controls can be configured for pressure sensitivity, which canbe used to accelerate user interactions. For example, a −/+ zoom controlmay want to accelerate zooming the harder the user presses. A continuouscontrol with a periodic interval may accelerate page turning, as theperiodic interval is automatically adjusted based on pressure.

typedef NS_ENUM(NSUInteger, NSSegmentSwitchTracking) {  NSSegmentSwitchTrackingMomentaryAcceleratorNS_ENUM_AVAILABLE_MAC(10_10_3) = 3, // accelerator behavior, onlyselected while tracking } /* This message is valid only fortrackingMode=NSSegmentSwitchTrackingMomentaryAccelerator and providesthe double value for the selected segment.  */ -(double)doubleValueForSelectedSegment NS_AVAILABLE_MAC(10_10_3);

The control may have its tracking mode set toNSSegmentSwitchTrackingMomentaryAccelerator.doubleValueForSelectedSegment represents the value which the individualsegment would return without any segment offset, as if it were astandalone accelerator button.

Accelerator Segmented Control Behaviors

Behavior of momentary accelerator segmented controls vary based on thestate of their continuous flag and whether or not the hardware supportspressure sensitivity. Below is a description of the behavior in eachconfiguration.

Continuous Momentary Accelerator, Non-Pressure Sensitive Trackpad:

Actions will be sent at the cell's periodic interval, starting after thecell's periodic delay. Upon mouse up a final action will be sent withselectedSegment=−1.

Continuous Momentary Accelerator, Pressure Sensitive Trackpad:

Actions will be sent at a periodic interval automatically adjusted basedon pressure, starting after the cell's delay. Upon mouse up a finalaction will be sent with selectedSegment=−1. In this configuration theadjusted periodic interval is useful for accelerated page turning ofcontent.

Non-Continuous Momentary Accelerator, Non-Pressure Sensitive Trackpad:

An initial action will be sent with doubleValueForSelectedSegment=1.0.Upon mouse up a final action will be sent withdoubleValueForSelectedSegment=0.0 and selectedSegment=−1.

Non-Continuous Momentary Accelerator, Pressure Sensitive Trackpad:

Each time the pressure changes, an action will be sent withdoubleValueForSelectedSegment=[1.0+pressure]. Upon mouse up a finalaction will be sent with doubleValueForSelectedSegment=0.0 andselectedSegment=−1. In this configuration thedoubleValueForSelectedSegment is useful to accelerate actions such aszooming.

What is claimed is:
 1. A method, comprising: at an electronic devicewith a touch-sensitive surface, wherein the electronic device includesone or more sensors to detect intensity of contacts with thetouch-sensitive surface and the electronic device stores a firstsoftware application: detecting a first touch input on thetouch-sensitive surface; in response to detecting the first touch inputon the touch-sensitive surface, determining a first intensity applied bythe first touch input on the touch-sensitive surface; identifying afirst intensity model identifier from a plurality of predefinedintensity model identifiers; in accordance with the first intensityapplied by the first touch input on the touch-sensitive surface and oneor more thresholds associated with the first intensity model identifier,determining a first touch characterization parameter; and, subsequent todetermining the first touch characterization parameter, sending firsttouch information to the first software application, wherein the firsttouch information includes the first intensity model identifier and thefirst touch characterization parameter.
 2. The method of claim 1,including: in response to detecting the first touch input on thetouch-sensitive surface, generating a tactile output in accordance withthe intensity applied by the first touch input on the touch-sensitivesurface and the one or more thresholds associated with the firstintensity model identifier.
 3. The method of claim 1, including:receiving one or more instructions from the first software applicationto generate a tactile output; and, in response to receiving the one ormore instructions from the software application, generating the tactileoutput in accordance with the one or more instructions from the softwareapplication.
 4. The method of claim 1, including: while continuing todetect the first touch input on the touch-sensitive surface: receivingone or more instructions to use an intensity model that corresponds to asecond intensity model identifier that is distinct from the firstintensity model identifier; and, subsequent to receiving the one or moreinstructions to use the intensity model that corresponds to the secondintensity model identifier: determining a second intensity applied bythe first touch input on the touch-sensitive surface; and processing thefirst touch input based on the second intensity model identifier.
 5. Themethod of claim 4, including: while continuing to detect the first touchinput on the touch-sensitive surface: subsequent to processing the firsttouch input based on the second intensity model identifier, receivingone or more instructions to use an intensity model that corresponds to athird intensity model identifier that is distinct from the firstintensity model identifier and the second intensity model identifier;and, subsequent to receiving the one or more instructions to use theintensity model that corresponds to the third intensity modelidentifier: determining a third intensity applied by the first touchinput on the touch-sensitive surface; and processing the first touchinput based on the third intensity model identifier.
 6. The method ofclaim 4, including: determining that the first touch input has ceased tobe detected on the touch-sensitive surface; and, subsequent todetermining that the first touch input has ceased to be detected on thetouch-sensitive surface: detecting a second touch input on thetouch-sensitive surface that is separate from the first touch input; andprocessing the second touch input based on the first intensity modelidentifier.
 7. The method of claim 4, wherein processing the touch inputbased on the second intensity model identifier includes: in accordancewith the second intensity applied by the touch input on thetouch-sensitive surface and one or more thresholds associated with thesecond intensity model identifier, determining a second touchcharacterization parameter, wherein the second touch characterizationparameter is distinct from the first touch characterization parameter;and, subsequent to determining the second touch characterizationparameter, sending second touch information to the first softwareapplication, wherein the second touch information includes the secondintensity model identifier and the second touch characterizationparameter.
 8. The method of claim 4, wherein processing the touch inputbased on the second intensity model identifier includes: foregoinggeneration of a tactile output in accordance with the second intensityfailing to satisfying the one or more thresholds associated with thesecond intensity model identifier, wherein the electronic device isconfigured to generate a tactile output in accordance with the secondintensity satisfying at least one of the one or more thresholdsassociated with the first intensity model identifier.
 9. The method ofclaim 1, including: subsequent to detecting the first touch input,receiving one or more instructions to use an intensity model thatcorresponds to a second intensity model identifier that is distinct fromthe first intensity model identifier; and, subsequent to receiving theone or more instructions to use the intensity model that corresponds tothe second intensity model identifier: detecting a second touch input onthe touch-sensitive surface; in response to detecting the second touchinput on the touch-sensitive surface, determining a second intensityapplied by the second touch input on the touch-sensitive surface; andprocessing the second touch input based on the second intensity modelidentifier.
 10. The method of claim 1, including: sending a stream ofintensity events to the first software application, each intensity eventcorresponding to an intensity applied by the touch input at acorresponding time.
 11. The method of claim 1, including selecting a setof thresholds, from a plurality of sets of thresholds, in accordancewith the identified intensity model identifier, and determining thetouch characterization parameter in accordance with the selected set ofthresholds.
 12. The method of claim 1, including identifying one or moreintensity ranges, in accordance with the identified intensity modelidentifier, and determining a touch characterization parameter inaccordance with the one or more identified intensity ranges.
 13. Themethod of claim 1, wherein identifying the first intensity modelidentifier includes identifying the first software application ascorresponding to the touch input and identifying an intensity modelidentifier registered by the first software application as the firstintensity model identifier.
 14. An electronic device, comprising: atouch-sensitive surface; one or more sensors to detect intensity ofcontacts with the touch-sensitive surface; one or more processors; andmemory storing one or more programs, wherein the one or more programsare configured to be executed by the one or more processors and a firstsoftware application, the one or more programs including instructionsfor: detecting a first touch input on the touch-sensitive surface; inresponse to detecting the first touch input on the touch-sensitivesurface, determining a first intensity applied by the first touch inputon the touch-sensitive surface; identifying a first intensity modelidentifier from a plurality of predefined intensity model identifiers;in accordance with the first intensity applied by the first touch inputon the touch-sensitive surface and one or more thresholds associatedwith the first intensity model identifier, determining a first touchcharacterization parameter; and, subsequent to determining the firsttouch characterization parameter, sending first touch information to thefirst software application, wherein the first touch information includesthe first intensity model identifier and the first touchcharacterization parameter.
 15. A non-transitory computer readablestorage medium storing one or more programs, the one or more programsincluding instructions, which, when executed by an electronic devicewith a touch-sensitive surface and one or more sensors to detectintensity of contacts with the touch-sensitive surface, cause the deviceto: detect a first touch input on the touch-sensitive surface; inresponse to detecting the first touch input on the touch-sensitivesurface, determine a first intensity applied by the first touch input onthe touch-sensitive surface; identify a first intensity model identifierfrom a plurality of predefined intensity model identifiers; inaccordance with the first intensity applied by the first touch input onthe touch-sensitive surface and one or more thresholds associated withthe first intensity model identifier, determine a first touchcharacterization parameter; and, subsequent to determining the firsttouch characterization parameter, send first touch information to afirst software application, wherein the first touch information includesthe first intensity model identifier and the first touchcharacterization parameter.